-- Logs begin at Wed 2022-08-10 11:49:06 UTC, end at Wed 2022-08-10 12:49:19 UTC. -- Aug 10 12:17:32.221949 dv-cinder-852261-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 12:17:32.231388 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 12:17:32 2022] *** Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: nodename: dv-cinder-852261-3-iscsi Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: machine: x86_64 Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: clock source: unix Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: pcre jit disabled Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: detected number of CPU cores: 4 Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: current working directory: / Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: detected binary path: /usr/bin/uwsgi-core Aug 10 12:17:32.233418 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: your processes number limit is 47680 Aug 10 12:17:32.234084 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: your memory page size is 4096 bytes Aug 10 12:17:32.234084 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: detected max file descriptor number: 2048 Aug 10 12:17:32.234084 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: lock engine: pthread robust mutexes Aug 10 12:17:32.234084 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: thunder lock: enabled Aug 10 12:17:32.234084 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 12:17:32.234084 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 12:17:32.272060 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: Python main interpreter initialized at 0x55aef20ff9c0 Aug 10 12:17:32.272060 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: python threads support enabled Aug 10 12:17:32.272060 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: your server socket listen backlog is limited to 100 connections Aug 10 12:17:32.272060 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: your mercy for graceful operations on workers is 90 seconds Aug 10 12:17:32.272403 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 12:17:32.272477 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: *** Operational MODE: preforking *** Aug 10 12:17:32.272743 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: *** uWSGI is running in multiple interpreter mode *** Aug 10 12:17:32.272743 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: spawned uWSGI master process (pid: 80051) Aug 10 12:17:32.272938 dv-cinder-852261-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 12:17:32.273732 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: spawned uWSGI worker 1 (pid: 80063, cores: 1) Aug 10 12:17:32.276115 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: spawned uWSGI worker 2 (pid: 80064, cores: 1) Aug 10 12:17:32.276115 dv-cinder-852261-3-iscsi devstack@keystone.service[80051]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 12:17:37.773850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:17:37.773850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: from cryptography.utils import int_from_bytes Aug 10 12:17:37.774625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:17:37.774625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: from cryptography.utils import int_from_bytes Aug 10 12:17:37.775206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:17:37.775206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: from cryptography.utils import int_from_bytes Aug 10 12:17:37.776142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 12:17:37.776142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: from cryptography.utils import int_from_bytes Aug 10 12:17:37.824357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 12:17:37.824549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 12:17:37.824652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 12:17:37.824743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] command line args: [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 12:17:37.824815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 12:17:37.824992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] command line args: [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 12:17:37.825071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 12:17:37.825176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ================================================================================ {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 12:17:37.825257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 12:17:37.825458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ================================================================================ {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 12:17:37.825545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] admin_token = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.825710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] config_dir = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.825789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] admin_token = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.825902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.825990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] config_dir = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] config_source = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] debug = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] config_source = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] debug = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.826934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] list_limit = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] list_limit = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.827958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] log_config_append = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] log_config_append = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] log_dir = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] log_dir = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] log_file = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] log_file = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.828997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.829958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.830907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.831924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.832264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.832426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.832517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] publish_errors = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.832611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] publish_errors = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.832684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.832850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.832927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.833902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] standard_threads = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] standard_threads = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] use_journal = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] use_json = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] use_journal = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] use_stderr = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.834977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] use_json = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.835072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] use_syslog = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.835151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] use_stderr = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.835287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] use_syslog = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.835371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.835524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 12:17:37.835603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.835774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.835851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.836998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.837082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.837251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.837416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.837526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.838299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.838392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.838561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.838642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.838813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.838911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.839050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.839129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.839300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.839444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.839531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.839762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.839842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.840015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.840099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.840252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.840450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.840607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.840754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.840833 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.841909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.842955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.843987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.844954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.845920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846833 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.846993 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.847928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.848936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.849920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.850861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.851971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.852937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.853951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.854030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.854453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.854535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.854703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.854784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.854940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.855971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.856915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.857904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.858981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.859990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.860941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] auth.external = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.861968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] auth.external = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] auth.password = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] auth.token = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.862952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] auth.password = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] auth.token = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.863903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] credential.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.864855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] credential.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.865854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.866983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.867062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.867237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.867325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.867504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.867583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.867748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.867836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.868835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.869843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.870965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.871938 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.872982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.873971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.874907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.875877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.876050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.876311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.876391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.876549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.876628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.876791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.876879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877833 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.877993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.878967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.879933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.880944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.881929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.882862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.883984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.884066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.884215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.884295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.884454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.885972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.886921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.887943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.888914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.889883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.890989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.891952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.892984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.893972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.894942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.895898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] role.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] role.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] role.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.896846 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] role.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.897851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.898992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.899981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.900115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.900289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.900924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.901178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.901258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.901451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.901542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.901701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.901791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.901951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.902032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.902185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.902310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.902542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.902631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.902796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.902877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.903985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.904970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.905950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.906917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.907970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.908931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.909931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.connection = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.connection = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.910934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.911865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.912981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.913956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.914037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.914191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.914279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.914354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 12:17:37.914434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.914665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 12:17:37.914839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80063) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 12:17:37.943690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.943821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.943915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.943993 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.944140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.944226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.954282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.954459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.982919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.983147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.983375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:37.983615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.035311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.035497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.085440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.085679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.086030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.086331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.151856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.151981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.152946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.153253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.153559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.176383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.176486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.176666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.176805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.176951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.177086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.177218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.177368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.177497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.177651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.177782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.177912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.178042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.178172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.188743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.188886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.201545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.201721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.201852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.201996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.202140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.202273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.202404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.202537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.202669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.202808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.202940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.203070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.203200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80063) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.203330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 12:17:38.220299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.220453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.220584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:17:38.220711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:17:38.220837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.220963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.221099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:17:38.221226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 12:17:38.221374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.221505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.221635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.221775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.221916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.222047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.222177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.222307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.234206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.234491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.234622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.234773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.234902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.235061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.235198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.235340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.242011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.242236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.245121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.245355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.245479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.245640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.245726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.245931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.246014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.246218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.246303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.246508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.246590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.246772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.246863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.247944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.248135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.270688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.271164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.272186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.272479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.272666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.272964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.277659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.277947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.278159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.278241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.278444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.278728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.284837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.285131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.285245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.285555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.289246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.289566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.289650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.289864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.289945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.290150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.290239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.290436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.290516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.290712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.290822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.291537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.291638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.291813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.291904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.292318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.308055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.308163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.308335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.308427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.312297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.312389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.315454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.315546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.315752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.315844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.320127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.320233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.323351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.323442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.326635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.326787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.326944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.327082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.327247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.327381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.327544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.327672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.327830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.327951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.328115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.328230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.344812 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.345109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.345414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.345689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.348783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.349075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.349374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.349643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.352766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.355181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.355468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.357439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.358320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.358612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.358924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.359008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.359228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.359325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.359513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.362067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.362353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.362652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.362951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.363232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.370977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.371271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.371560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.371844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.372127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.372407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.372687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.374660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.374946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.375229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.375513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.375797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.376074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.376352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.389175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.392165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.392703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.395135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.395677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.396726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.398249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.398554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.398637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.398843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.400722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.402294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.402543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.402834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.405799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.407443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.409244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.410391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.410691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.410861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.410991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.411280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.413798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.414093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.414376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.414651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.420073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.420372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.420664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.423166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.423455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.423736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.427569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80063) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.427859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.428149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.428432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.428707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.428992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.429289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.429583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.429973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.430254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.430532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.430809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.430890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 12:17:38.431095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.431175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.431363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80063) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.431453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.431707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.431983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.432264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.432543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.432828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.433207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.433506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.433781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.434051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.434320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.434592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 12:17:38.465136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55aef20ff9c0 pid: 80063 (default app) Aug 10 12:17:38.468130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55aef20ff9c0 pid: 80064 (default app) Aug 10 12:17:38.468407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddaff8-ff15-470f-9b47-960a9bf6c622 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:38.468696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddaff8-ff15-470f-9b47-960a9bf6c622 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:38.468970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddaff8-ff15-470f-9b47-960a9bf6c622 None None] PATH_INFO: `/v3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:38.473393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Wed Aug 10 12:17:38 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:46.032640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6cef-621f-45e4-b71f-33049701f0eb None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:46.033234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6cef-621f-45e4-b71f-33049701f0eb None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:46.033234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6cef-621f-45e4-b71f-33049701f0eb None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:46.037591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1/2] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:17:46.042658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:46.043050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:46.043204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:46.112951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: INFO dbcounter [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] Registered counter for database keystone Aug 10 12:17:46.132264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG oslo_db.sqlalchemy.engines [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80063) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 12:17:46.138877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writer thread running {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 12:17:46.318986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG passlib.handlers.bcrypt [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80063) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 12:17:46.319324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG passlib.handlers.bcrypt [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80063) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 12:17:46.374183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:46.375782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:46.503145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 2/3] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 12:17:46.637543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d6cef-621f-45e4-b71f-33049701f0eb None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:46.710874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: INFO dbcounter [None req-bd4d6cef-621f-45e4-b71f-33049701f0eb None None] Registered counter for database keystone Aug 10 12:17:46.731653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG oslo_db.sqlalchemy.engines [None req-bd4d6cef-621f-45e4-b71f-33049701f0eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80064) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 12:17:46.737759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writer thread running {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 12:17:46.855078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d6cef-621f-45e4-b71f-33049701f0eb None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:46.859480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810b66-6e48-4eae-87de-ce0aa2779279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:46.860091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810b66-6e48-4eae-87de-ce0aa2779279 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:46.860368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810b66-6e48-4eae-87de-ce0aa2779279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:46.860636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810b66-6e48-4eae-87de-ce0aa2779279 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:46.868553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810b66-6e48-4eae-87de-ce0aa2779279 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:46.920531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING keystone.common.rbac_enforcer.enforcer [None req-24810b66-6e48-4eae-87de-ce0aa2779279 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 12:17:46.920871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810b66-6e48-4eae-87de-ce0aa2779279 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:46.930173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 2/4] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 10 12:17:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 12:17:46.936800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:46.951298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0419d1-1c49-467e-be03-7750fa1afe78 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:46.955742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60090fe3-d239-466a-b56e-ffcd46ae3c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:46.956845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60090fe3-d239-466a-b56e-ffcd46ae3c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:46.957130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60090fe3-d239-466a-b56e-ffcd46ae3c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:46.957438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60090fe3-d239-466a-b56e-ffcd46ae3c4e admin admin] PATH_INFO: `/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:46.958888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60090fe3-d239-466a-b56e-ffcd46ae3c4e admin admin] RBAC: Authorizing `identity:get_project(project_id=bd2efc376f4648d0a2b40e0c73cde3c5)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:47.010537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING keystone.common.rbac_enforcer.enforcer [None req-60090fe3-d239-466a-b56e-ffcd46ae3c4e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 12:17:47.010884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60090fe3-d239-466a-b56e-ffcd46ae3c4e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:47.015497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 3/5] 10.222.0.65 () {56 vars in 1207 bytes} [Wed Aug 10 12:17:46 2022] GET /identity/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:48.971110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903ba1-a07f-41f4-94ec-a28d852874a1 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:48.971642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903ba1-a07f-41f4-94ec-a28d852874a1 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:48.971642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903ba1-a07f-41f4-94ec-a28d852874a1 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:48.975733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 3/6] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:17:48.981986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d7e4c-61a2-4d3e-a7ad-a5b13e93fe9a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:48.982401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d7e4c-61a2-4d3e-a7ad-a5b13e93fe9a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:48.982567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d7e4c-61a2-4d3e-a7ad-a5b13e93fe9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:49.003253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e28d7e4c-61a2-4d3e-a7ad-a5b13e93fe9a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:49.004761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e28d7e4c-61a2-4d3e-a7ad-a5b13e93fe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:49.012911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 4/7] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 12:17:49.034437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb903ba1-a07f-41f4-94ec-a28d852874a1 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:49.046224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb903ba1-a07f-41f4-94ec-a28d852874a1 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:49.050417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05271169-af84-42cc-9bb0-d39d25320ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:49.051001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05271169-af84-42cc-9bb0-d39d25320ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:49.051285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05271169-af84-42cc-9bb0-d39d25320ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:49.051554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05271169-af84-42cc-9bb0-d39d25320ea8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:49.059046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05271169-af84-42cc-9bb0-d39d25320ea8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:49.060811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05271169-af84-42cc-9bb0-d39d25320ea8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:49.069367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 4/8] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 10 12:17:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:17:49.074794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d7e4c-61a2-4d3e-a7ad-a5b13e93fe9a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:49.083490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d7e4c-61a2-4d3e-a7ad-a5b13e93fe9a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:49.088301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ecf31-68ea-480a-9514-6aaf572c3550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:49.088901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ecf31-68ea-480a-9514-6aaf572c3550 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:49.089178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ecf31-68ea-480a-9514-6aaf572c3550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:49.089478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ecf31-68ea-480a-9514-6aaf572c3550 admin admin] PATH_INFO: `/v3/users/474f14ac319744fba24161f36fcf9aa3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:49.090775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ecf31-68ea-480a-9514-6aaf572c3550 admin admin] RBAC: Authorizing `identity:get_user(user_id=474f14ac319744fba24161f36fcf9aa3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:49.092568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ecf31-68ea-480a-9514-6aaf572c3550 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:49.094072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 5/9] 10.222.0.65 () {56 vars in 1198 bytes} [Wed Aug 10 12:17:49 2022] GET /identity/v3/users/474f14ac319744fba24161f36fcf9aa3 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:51.148212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ca39c-7f86-4d32-ad9b-d857afa96cd3 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.148752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ca39c-7f86-4d32-ad9b-d857afa96cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.148752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ca39c-7f86-4d32-ad9b-d857afa96cd3 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.149098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605341be-f50e-461b-abd7-fb05a1965bdc None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.149476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605341be-f50e-461b-abd7-fb05a1965bdc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.149694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605341be-f50e-461b-abd7-fb05a1965bdc None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.150154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 5/10] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:17:51.151231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 6/11] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:17:51.154763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.155054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.155324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.156051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0ae7-d908-4136-9b13-0230a36a91d7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.156339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0ae7-d908-4136-9b13-0230a36a91d7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.156620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0ae7-d908-4136-9b13-0230a36a91d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.174070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-34fb0ae7-d908-4136-9b13-0230a36a91d7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:51.175235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-34fb0ae7-d908-4136-9b13-0230a36a91d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:51.198358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG passlib.handlers.bcrypt [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80064) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 12:17:51.198789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG passlib.handlers.bcrypt [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80064) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 12:17:51.244117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:51.245739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:51.254774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 7/12] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:17:51.278036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0ae7-d908-4136-9b13-0230a36a91d7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.301608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:51.344479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 6/13] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:17:51.366434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.381412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0ae7-d908-4136-9b13-0230a36a91d7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:51.414400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:51.415027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.415814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.416083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.426775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:17:51.435992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:17:51.436978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:17:51.437852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 8/14] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:17:51.444560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.465077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ead9d8-31db-4a16-90c8-7b23b0f6f6b5 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:51.467456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:51.504116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:51.504960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.505242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.505539 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.524728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:51.543270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507c3c0-90bf-492d-9269-2f5d4cc19a1f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:51.571762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:51.573348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:51.580178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:51.580837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.581122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.581421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.582095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:17:51.582393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:51.583768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:51.597436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 9/15] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:17:51.603346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.676616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 7/16] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 12:17:51.682766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.685992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:51.785198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e523fb-373c-41f8-b0e5-6d47b3e8b0ec None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:51.806754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8c22-27d4-4215-a269-6cd6a7a12e89 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:51.818287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:51.819126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.819405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.819674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.837880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:51.839339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:51.839874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:51.840476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.840756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.841032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] PATH_INFO: `/v3/users/474f14ac319744fba24161f36fcf9aa3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.842210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] RBAC: Authorizing `identity:get_user(user_id=474f14ac319744fba24161f36fcf9aa3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:51.843654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:51.845462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 10/17] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/users/474f14ac319744fba24161f36fcf9aa3 => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:51.850612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:17:51.858362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:17:51.859206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:17:51.859999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:17:51.859999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:17:51.859999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:17:51.859999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 8/18] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:17:51.865022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.866425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:51.921986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:51.950364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813a5e0-08ab-47a0-bc78-93c19e24fd80 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:51.964232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e6593-7d45-40fa-9cdf-6ebdb38b1873 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:51.983494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:51.984124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.984409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.984675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.985812 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:51.987247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:51.988942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 11/19] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:17:51.994454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:51.997996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d971519-48cf-46d1-9273-3345878f7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:51.998602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d971519-48cf-46d1-9273-3345878f7f17 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:51.998886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d971519-48cf-46d1-9273-3345878f7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:51.999158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d971519-48cf-46d1-9273-3345878f7f17 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:51.999806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d971519-48cf-46d1-9273-3345878f7f17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:17:52.000114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d971519-48cf-46d1-9273-3345878f7f17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:52.001532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d971519-48cf-46d1-9273-3345878f7f17 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:52.019167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 9/20] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:17:52.077996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:52.099049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e8205-05f6-4e39-a48b-b014671353ba None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:52.134903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b6a1c-077d-4cf9-95b3-7c6b05def4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:52.135550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b6a1c-077d-4cf9-95b3-7c6b05def4bc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:52.135836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b6a1c-077d-4cf9-95b3-7c6b05def4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:52.136113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b6a1c-077d-4cf9-95b3-7c6b05def4bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:52.136856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b6a1c-077d-4cf9-95b3-7c6b05def4bc None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.domain.id=default, user.id=474f14ac319744fba24161f36fcf9aa3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:17:52.137167 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b6a1c-077d-4cf9-95b3-7c6b05def4bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:52.138680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b6a1c-077d-4cf9-95b3-7c6b05def4bc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:52.145992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 12/21] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 10 12:17:51 2022] GET /identity/v3/role_assignments?user.id=474f14ac319744fba24161f36fcf9aa3&scope.domain.id=default&role.id=2470da926a0342698ca115a00c86fea3 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:17:54.374627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4931283-c090-4fb1-bd6f-998e655efe66 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.375588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4931283-c090-4fb1-bd6f-998e655efe66 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.376086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4931283-c090-4fb1-bd6f-998e655efe66 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.377673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 10/22] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:17:54.390610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dcf04-4ad1-424f-a1dd-5ffedf74e859 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.391497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dcf04-4ad1-424f-a1dd-5ffedf74e859 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.391991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dcf04-4ad1-424f-a1dd-5ffedf74e859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.422221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-016dcf04-4ad1-424f-a1dd-5ffedf74e859 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:54.423757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-016dcf04-4ad1-424f-a1dd-5ffedf74e859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:54.506035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 13/23] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:17:54.540704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4931283-c090-4fb1-bd6f-998e655efe66 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:54.645315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a4523-f1ee-4b00-8ba6-5289fb16f6ce None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.645587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a4523-f1ee-4b00-8ba6-5289fb16f6ce None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.645803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a4523-f1ee-4b00-8ba6-5289fb16f6ce None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.647501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 14/24] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:17:54.652190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a955-69bc-4042-942a-5c74fa9dc5eb None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.652477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a955-69bc-4042-942a-5c74fa9dc5eb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.652752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a955-69bc-4042-942a-5c74fa9dc5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.678349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2f24a955-69bc-4042-942a-5c74fa9dc5eb None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:54.681522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2f24a955-69bc-4042-942a-5c74fa9dc5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:54.694294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4931283-c090-4fb1-bd6f-998e655efe66 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:54.718782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:54.758882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87603e9-91b8-428b-b123-f5281e422c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:54.760383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87603e9-91b8-428b-b123-f5281e422c0a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.760860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87603e9-91b8-428b-b123-f5281e422c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.761342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87603e9-91b8-428b-b123-f5281e422c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.780369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87603e9-91b8-428b-b123-f5281e422c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:54.782870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87603e9-91b8-428b-b123-f5281e422c0a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:54.817308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 15/25] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:17:54.819993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee398d-5399-4643-87c5-bc3b1f13f03f None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.820474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee398d-5399-4643-87c5-bc3b1f13f03f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.820893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee398d-5399-4643-87c5-bc3b1f13f03f None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.826992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 16/26] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:17:54.828671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79403954-4322-4dd7-8c0c-813b561647bf None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.829105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79403954-4322-4dd7-8c0c-813b561647bf None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.829569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79403954-4322-4dd7-8c0c-813b561647bf None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.831099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 17/27] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:17:54.838818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e269c8-1be9-4a6c-b2cb-6b744eb52df5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.839325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e269c8-1be9-4a6c-b2cb-6b744eb52df5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.839763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e269c8-1be9-4a6c-b2cb-6b744eb52df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.862913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:54.873403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-49e269c8-1be9-4a6c-b2cb-6b744eb52df5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:54.874652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-49e269c8-1be9-4a6c-b2cb-6b744eb52df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a87603e9-91b8-428b-b123-f5281e422c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:17:54.918341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:17:54.919294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:17:54.920091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:17:54.920091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:17:54.920091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:17:54.920091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 11/28] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:17:54.921026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3b9d-d224-420d-88e7-261635c8d510 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:54.921443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3b9d-d224-420d-88e7-261635c8d510 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:54.921814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3b9d-d224-420d-88e7-261635c8d510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:54.940468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6d3e3b9d-d224-420d-88e7-261635c8d510 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:54.941832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6d3e3b9d-d224-420d-88e7-261635c8d510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:54.948835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 18/29] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:17:54.949793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e269c8-1be9-4a6c-b2cb-6b744eb52df5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.020703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 12/30] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:17:55.021741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e3b9d-d224-420d-88e7-261635c8d510 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.050812 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e269c8-1be9-4a6c-b2cb-6b744eb52df5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.087488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:55.088143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.088428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.088705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.089433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:55.091142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:55.101793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:55.128683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:55.132981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e3b9d-d224-420d-88e7-261635c8d510 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.171433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:55.172067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.172423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.172772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.173589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:17:55.173968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:55.175493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:55.188104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 13/31] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:55.189024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.225660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 19/32] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 10 12:17:54 2022] POST /identity/v3/roles => generated 219 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:17:55.226999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.299813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3432a-e31f-4734-b8df-6ae451036069 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.338247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:55.339031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.339384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.339737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.361300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:17:55.362326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:17:55.363176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 14/33] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:17:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:17:55.364407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.374202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a500-9c45-43a6-b06c-0c3df82e383e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.409632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:55.446725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:55.447365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.447554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.447834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.449070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:55.450653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:55.473542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 20/34] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 10 12:17:55 2022] POST /identity/v3/roles => generated 225 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:17:55.475075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.539633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5e761-e17d-4f2e-8403-0c22fece6e98 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.551614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:55.586376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:55.587124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.587510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.587776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.588727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:55.590471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:55.612646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf12ab-a86a-4ba3-bd8d-3404cb1dad35 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.653222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:55.654248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.654810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.655581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.656527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:17:55.657047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:55.658759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:55.668770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 21/35] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:55.674488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.760529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 15/36] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:17:55 2022] POST /identity/v3/projects => generated 301 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:17:55.792674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03db8c-7339-4875-818a-9a5f1cbd627f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.814676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:55.836001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bde97-34d8-4eab-8f62-51d0c090e27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:55.836668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bde97-34d8-4eab-8f62-51d0c090e27c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.836950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bde97-34d8-4eab-8f62-51d0c090e27c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.837281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bde97-34d8-4eab-8f62-51d0c090e27c None admin] PATH_INFO: `/v3/users/474f14ac319744fba24161f36fcf9aa3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.838499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bde97-34d8-4eab-8f62-51d0c090e27c None admin] RBAC: Authorizing `identity:get_user(user_id=474f14ac319744fba24161f36fcf9aa3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:55.840004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bde97-34d8-4eab-8f62-51d0c090e27c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:55.842183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 22/37] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:17:55 2022] GET /identity/v3/users/474f14ac319744fba24161f36fcf9aa3 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:55.847614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:55.872315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e652b-3aa4-451e-9ba8-7a450dc87700 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.872604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e652b-3aa4-451e-9ba8-7a450dc87700 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.872888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e652b-3aa4-451e-9ba8-7a450dc87700 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.874272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 23/38] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:17:55.880388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79830-9002-4f8e-95e7-a3c7a0f22bf3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:55.880684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79830-9002-4f8e-95e7-a3c7a0f22bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:55.880964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79830-9002-4f8e-95e7-a3c7a0f22bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:55.900573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a7d79830-9002-4f8e-95e7-a3c7a0f22bf3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:55.902301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a7d79830-9002-4f8e-95e7-a3c7a0f22bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:55.946609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:55.986652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95345ecf-3f72-4b03-9485-6f1ad1e24ff5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:55.998601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 24/39] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:17:56.020673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79830-9002-4f8e-95e7-a3c7a0f22bf3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:56.022942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:56.023726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:56.024081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:56.024365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:56.025710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:56.027300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:56.029038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 16/40] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:17:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:56.034941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:56.135646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79830-9002-4f8e-95e7-a3c7a0f22bf3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:56.147016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2476-1819-49f9-b06b-0340b269d98f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:56.150200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:56.174966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12cbaf-ff1f-4813-b44d-502355cbcbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:56.175762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12cbaf-ff1f-4813-b44d-502355cbcbbd None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:56.176049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12cbaf-ff1f-4813-b44d-502355cbcbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:56.176628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12cbaf-ff1f-4813-b44d-502355cbcbbd None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:56.177577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12cbaf-ff1f-4813-b44d-502355cbcbbd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:56.179411 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12cbaf-ff1f-4813-b44d-502355cbcbbd None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:56.183927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:56.184601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:56.184889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:56.185171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] PATH_INFO: `/v3/domains/default/users/474f14ac319744fba24161f36fcf9aa3/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:56.187890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=474f14ac319744fba24161f36fcf9aa3, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:56.189472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:56.196059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 25/41] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 10 12:17:56 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:17:56.198756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08403838-1c81-4362-83d5-b71d390f049d None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:56.199065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08403838-1c81-4362-83d5-b71d390f049d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:56.199369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08403838-1c81-4362-83d5-b71d390f049d None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:56.200841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 26/42] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:17:56.201766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:56.207490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 17/43] 10.222.0.65 () {58 vars in 1361 bytes} [Wed Aug 10 12:17:56 2022] PUT /identity/v3/domains/default/users/474f14ac319744fba24161f36fcf9aa3/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:17:56.207943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3ece-4e86-432d-b73c-334ee0fdaddc None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:56.208680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3ece-4e86-432d-b73c-334ee0fdaddc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:56.208969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3ece-4e86-432d-b73c-334ee0fdaddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:56.230161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-bf6f3ece-4e86-432d-b73c-334ee0fdaddc None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:56.231434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bf6f3ece-4e86-432d-b73c-334ee0fdaddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:56.312806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 27/44] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:17:56.336287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:56.443050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5e070-a0b4-4f18-a17e-db7bf3ff77ab None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:56.476304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6740f-06d8-4696-b6b4-449d849d3ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:56.476931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6740f-06d8-4696-b6b4-449d849d3ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:56.477354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6740f-06d8-4696-b6b4-449d849d3ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:56.477691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6740f-06d8-4696-b6b4-449d849d3ff1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:56.479568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6740f-06d8-4696-b6b4-449d849d3ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:56.481016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6740f-06d8-4696-b6b4-449d849d3ff1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:56.482597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 18/45] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:17:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:56.488450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f3ece-4e86-432d-b73c-334ee0fdaddc None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:56.588505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f3ece-4e86-432d-b73c-334ee0fdaddc None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:56.590763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:56.625161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bce14-5db0-4a57-9714-4839f3344047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:56.625788 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bce14-5db0-4a57-9714-4839f3344047 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:56.626064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bce14-5db0-4a57-9714-4839f3344047 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:56.626332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bce14-5db0-4a57-9714-4839f3344047 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:56.626979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bce14-5db0-4a57-9714-4839f3344047 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:56.628436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bce14-5db0-4a57-9714-4839f3344047 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:56.657246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 28/46] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:17:56 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:17:58.208690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac315bff-2fa8-4358-9fcd-7fa70accab55 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.209245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac315bff-2fa8-4358-9fcd-7fa70accab55 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.209245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac315bff-2fa8-4358-9fcd-7fa70accab55 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.210673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 19/47] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:17:58.216371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a6315-6bab-4203-a834-fdfee674e8cb None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.216779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a6315-6bab-4203-a834-fdfee674e8cb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.216928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a6315-6bab-4203-a834-fdfee674e8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.235087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-261a6315-6bab-4203-a834-fdfee674e8cb None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:58.236260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-261a6315-6bab-4203-a834-fdfee674e8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:58.302832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 29/48] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:17:58.325651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac315bff-2fa8-4358-9fcd-7fa70accab55 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:58.393329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:58.430419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac315bff-2fa8-4358-9fcd-7fa70accab55 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:58.465319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942ac3b-3a21-4a30-9779-6c9d58b14fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:58.465885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ac3b-3a21-4a30-9779-6c9d58b14fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.466177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ac3b-3a21-4a30-9779-6c9d58b14fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.466452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ac3b-3a21-4a30-9779-6c9d58b14fc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.476507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942ac3b-3a21-4a30-9779-6c9d58b14fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a942ac3b-3a21-4a30-9779-6c9d58b14fc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:17:58.481643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:17:58.482495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:17:58.483240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 20/49] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:17:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:17:58.488616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a6315-6bab-4203-a834-fdfee674e8cb None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:58.570803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:58.588745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a6315-6bab-4203-a834-fdfee674e8cb None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:58.622988 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:58.623611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.623888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.624161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.624836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:17:58.625143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:58.625690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecd72c-7eaf-4ddc-92ad-802d56eceb96 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.625981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecd72c-7eaf-4ddc-92ad-802d56eceb96 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.626257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecd72c-7eaf-4ddc-92ad-802d56eceb96 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.626571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:58.627612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 21/50] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:17:58.632450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1d4e0-b2eb-4742-a31d-d82a6429bf3d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.632732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1d4e0-b2eb-4742-a31d-d82a6429bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.633008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1d4e0-b2eb-4742-a31d-d82a6429bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.639583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 30/51] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:17:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:17:58.644935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:58.651877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f8e1d4e0-b2eb-4742-a31d-d82a6429bf3d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:17:58.653439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f8e1d4e0-b2eb-4742-a31d-d82a6429bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:17:58.718250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 22/52] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:17:58.740735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1d4e0-b2eb-4742-a31d-d82a6429bf3d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:58.744052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83685d-2df7-40bf-8fdc-35d0f3c7af35 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:58.777834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:58.778459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.778737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.779007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] PATH_INFO: `/v3/users/474f14ac319744fba24161f36fcf9aa3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.780136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] RBAC: Authorizing `identity:get_user(user_id=474f14ac319744fba24161f36fcf9aa3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:58.781620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:58.783165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 31/53] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:17:58 2022] GET /identity/v3/users/474f14ac319744fba24161f36fcf9aa3 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:17:58.788407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:58.796714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:58.807214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:58.840369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1d4e0-b2eb-4742-a31d-d82a6429bf3d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:58.874953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:58.875581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.875863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.876136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.877254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:58.878744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:58.880317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 23/54] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:17:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:17:58.886306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:58.887547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e149e-3b9d-4fd5-aef9-ffd76400990e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:58.921116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:58.921761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:58.922040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:58.922309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:58.923515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:58.925482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:58.927530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 32/55] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:17:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:17:58.933792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:17:58.983661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effbbd1-1c30-44ae-9411-cdceaa547ab0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:59.006984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:59.018343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c63de-3091-47b4-ab4a-65ea39f77403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:59.018971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c63de-3091-47b4-ab4a-65ea39f77403 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:59.019264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c63de-3091-47b4-ab4a-65ea39f77403 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:59.019539 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c63de-3091-47b4-ab4a-65ea39f77403 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:59.020175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c63de-3091-47b4-ab4a-65ea39f77403 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:59.021677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c63de-3091-47b4-ab4a-65ea39f77403 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:59.022077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:17:59.038039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27905c-677d-47c6-b74a-4b2a61f8310f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:17:59.048207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 24/56] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 10 12:17:58 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:17:59.074348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83bc43-bb55-4202-b490-3e68479fb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:17:59.074990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83bc43-bb55-4202-b490-3e68479fb279 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:17:59.075275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83bc43-bb55-4202-b490-3e68479fb279 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:17:59.075551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83bc43-bb55-4202-b490-3e68479fb279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:17:59.076303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83bc43-bb55-4202-b490-3e68479fb279 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.domain.id=default, user.id=474f14ac319744fba24161f36fcf9aa3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:17:59.076614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83bc43-bb55-4202-b490-3e68479fb279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:17:59.078152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83bc43-bb55-4202-b490-3e68479fb279 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:17:59.084920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 33/57] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 10 12:17:58 2022] GET /identity/v3/role_assignments?user.id=474f14ac319744fba24161f36fcf9aa3&scope.domain.id=default&role.id=2470da926a0342698ca115a00c86fea3 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:01.068136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1c1c-9f2d-4c09-b051-972275f68686 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:01.068646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1c1c-9f2d-4c09-b051-972275f68686 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:01.068727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1c1c-9f2d-4c09-b051-972275f68686 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:01.070085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 25/58] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:01.075784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db873c1-a36f-402c-b388-b6ad3386d0d7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:01.076158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db873c1-a36f-402c-b388-b6ad3386d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:01.076369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db873c1-a36f-402c-b388-b6ad3386d0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:01.094463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2db873c1-a36f-402c-b388-b6ad3386d0d7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:01.095625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2db873c1-a36f-402c-b388-b6ad3386d0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:01.159148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 34/59] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:01.182744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1c1c-9f2d-4c09-b051-972275f68686 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:01.286102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1c1c-9f2d-4c09-b051-972275f68686 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:01.318298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86465e2-56b9-4e74-8582-43ce8d0c516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:01.318858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86465e2-56b9-4e74-8582-43ce8d0c516e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:01.319140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86465e2-56b9-4e74-8582-43ce8d0c516e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:01.319417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86465e2-56b9-4e74-8582-43ce8d0c516e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:01.320529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86465e2-56b9-4e74-8582-43ce8d0c516e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:01.322522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86465e2-56b9-4e74-8582-43ce8d0c516e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:01.324075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 26/60] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:01.329556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db873c1-a36f-402c-b388-b6ad3386d0d7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:01.404505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:01.424979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db873c1-a36f-402c-b388-b6ad3386d0d7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:01.458282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91031-0caa-4dfb-8fd5-c8151e58f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:01.458903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91031-0caa-4dfb-8fd5-c8151e58f777 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:01.459179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91031-0caa-4dfb-8fd5-c8151e58f777 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:01.459448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91031-0caa-4dfb-8fd5-c8151e58f777 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:01.460086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b91031-0caa-4dfb-8fd5-c8151e58f777 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:01.461565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b91031-0caa-4dfb-8fd5-c8151e58f777 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:01.487442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 35/61] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:01 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:04.611733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ead4-4a80-4198-a9e7-03a490d1f591 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:04.612196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ead4-4a80-4198-a9e7-03a490d1f591 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:04.612268 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ead4-4a80-4198-a9e7-03a490d1f591 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:04.613948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 27/62] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:04.619274 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858b825-8425-48c9-957b-36540b448fd7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:04.620071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858b825-8425-48c9-957b-36540b448fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:04.620536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858b825-8425-48c9-957b-36540b448fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:04.642030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6858b825-8425-48c9-957b-36540b448fd7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:04.643544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6858b825-8425-48c9-957b-36540b448fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:04.722917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79c151-6c66-4a10-b984-2b70e1d23fae None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:04.723530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79c151-6c66-4a10-b984-2b70e1d23fae None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:04.724036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79c151-6c66-4a10-b984-2b70e1d23fae None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:04.725587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 28/63] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:18:04.731162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34b0d7-7903-48c8-836c-e10eb98a1f97 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:04.731678 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34b0d7-7903-48c8-836c-e10eb98a1f97 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:04.732145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34b0d7-7903-48c8-836c-e10eb98a1f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:04.740724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 36/64] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:04.762311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6a34b0d7-7903-48c8-836c-e10eb98a1f97 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:04.763940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6a34b0d7-7903-48c8-836c-e10eb98a1f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:04.764604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858b825-8425-48c9-957b-36540b448fd7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:04.833720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:04.858685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:04.871503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858b825-8425-48c9-957b-36540b448fd7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:04.896090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 29/65] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:04.897818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1949d-05be-40fd-bc7e-272064739ea7 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:04.898258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1949d-05be-40fd-bc7e-272064739ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:04.898760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1949d-05be-40fd-bc7e-272064739ea7 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:04.900561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 30/66] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:04.906076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8654-111b-49ed-8b13-9169ffd1f8ed None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:04.906811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8654-111b-49ed-8b13-9169ffd1f8ed None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:04.907259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8654-111b-49ed-8b13-9169ffd1f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:04.910559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:04.911318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:04.911697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:04.912069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:04.923248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:04.928527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-953d8654-111b-49ed-8b13-9169ffd1f8ed None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:04.928867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:04.929921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:04.931492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:04.933166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 37/67] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:04.933555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-953d8654-111b-49ed-8b13-9169ffd1f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:04.937506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.033316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 31/68] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:05.034835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8654-111b-49ed-8b13-9169ffd1f8ed None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.052252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdba55-b8ce-4a70-95f2-831e2a7eff6a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.077519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.089839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.090291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.090668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.091030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.103492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:05.108801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:05.109694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:05.110449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 38/69] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:05.111057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.156240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8654-111b-49ed-8b13-9169ffd1f8ed None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.158540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.189759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5d0fd-6db3-4537-afa9-5ff024fde1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.190363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5d0fd-6db3-4537-afa9-5ff024fde1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.190648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5d0fd-6db3-4537-afa9-5ff024fde1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.190917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5d0fd-6db3-4537-afa9-5ff024fde1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.191646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5d0fd-6db3-4537-afa9-5ff024fde1e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:05.192237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5d0fd-6db3-4537-afa9-5ff024fde1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:05.193646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5d0fd-6db3-4537-afa9-5ff024fde1e6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:05.205650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 32/70] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:05.210196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf4dfb-5185-47dd-a87e-355268eda5a6 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.210196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf4dfb-5185-47dd-a87e-355268eda5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.210196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf4dfb-5185-47dd-a87e-355268eda5a6 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.210196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 33/71] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:05.210196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf4dfb-5185-47dd-a87e-355268eda5a6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.216970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da32f7-e4c5-4d91-812d-2f19a0f03ab0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.249523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8edbc-9318-4ae2-805b-3afa8da26571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.250144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8edbc-9318-4ae2-805b-3afa8da26571 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.250421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8edbc-9318-4ae2-805b-3afa8da26571 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.250691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8edbc-9318-4ae2-805b-3afa8da26571 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.260732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8edbc-9318-4ae2-805b-3afa8da26571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-eca8edbc-9318-4ae2-805b-3afa8da26571 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:05.267275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:05.268227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:05.269370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 39/72] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:05.270202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb485fb-3e4c-4d46-b071-2679653e2bba None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.270202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb485fb-3e4c-4d46-b071-2679653e2bba None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.270202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb485fb-3e4c-4d46-b071-2679653e2bba None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.271193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 40/73] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:05.275628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb485fb-3e4c-4d46-b071-2679653e2bba None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.317994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.319355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf4dfb-5185-47dd-a87e-355268eda5a6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.351927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc76c5-24bd-40ef-9a38-74bc9785563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.353107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc76c5-24bd-40ef-9a38-74bc9785563b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.353418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc76c5-24bd-40ef-9a38-74bc9785563b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.353957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc76c5-24bd-40ef-9a38-74bc9785563b None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.354615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc76c5-24bd-40ef-9a38-74bc9785563b None admin] RBAC: Adding query filter params (name=member) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:05.354921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc76c5-24bd-40ef-9a38-74bc9785563b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:05.356292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc76c5-24bd-40ef-9a38-74bc9785563b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:05.364218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 34/74] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:05.365343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab4b5-28f8-4ca7-b2a0-fdc4cba782a7 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.365636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab4b5-28f8-4ca7-b2a0-fdc4cba782a7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.365915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab4b5-28f8-4ca7-b2a0-fdc4cba782a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.379341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb485fb-3e4c-4d46-b071-2679653e2bba None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.382610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a34ab4b5-28f8-4ca7-b2a0-fdc4cba782a7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:05.383816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a34ab4b5-28f8-4ca7-b2a0-fdc4cba782a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:05.407001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.413173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.413849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.414153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.414435 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] PATH_INFO: `/v3/users/474f14ac319744fba24161f36fcf9aa3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.416172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] RBAC: Authorizing `identity:get_user(user_id=474f14ac319744fba24161f36fcf9aa3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:05.417689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:05.419262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 41/75] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/users/474f14ac319744fba24161f36fcf9aa3 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:05.423841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.451119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 35/76] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:05.452355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2952a2-e677-42a0-ac7f-4f894a6ee909 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.452646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2952a2-e677-42a0-ac7f-4f894a6ee909 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.452922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2952a2-e677-42a0-ac7f-4f894a6ee909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.470511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-aa2952a2-e677-42a0-ac7f-4f894a6ee909 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:05.471689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-aa2952a2-e677-42a0-ac7f-4f894a6ee909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:05.520826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218201cc-577d-47bd-96e1-9e0f9421452f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.533813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.539456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 36/77] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:05.540299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2952a2-e677-42a0-ac7f-4f894a6ee909 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.565606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.566224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.566505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.566783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.567447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:05.567751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:05.569159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:05.578285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 42/78] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:05.579504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.629524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.642727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2952a2-e677-42a0-ac7f-4f894a6ee909 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.676079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.676714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.676845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55091d1-1578-4e9b-8ae6-3a2335e8af75 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.677244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.677244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.692698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:05.694204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:05.695831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 37/79] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:05.697135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.710607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.711222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.711506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.712382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.754013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.780202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:05.781701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:05.794772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8d92b-5887-4c5b-8714-8806cb45ec92 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.830223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.830850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.831134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.831685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.841810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:05.846463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:05.847291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:05.848068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 38/80] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:05.848673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.856919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.885060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 43/81] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:05.885857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:05.946801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc08d56-4df3-497b-8cdf-7f88115dcf38 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.958585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:05.980092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:05.980724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:05.980999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:05.981288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:05.982411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:05.983869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:05.984076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d956872-8c28-45b6-8e4f-d27c082ebce0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:05.985761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 39/82] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:05.986878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.017603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.018230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.018508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.018780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.019897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.021377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.022927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 44/83] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:06.027178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.064208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.088786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df9595-5e67-481e-bebf-a5cf982043a6 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.121348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd764d-697b-46b8-a933-a2b0c4f69f2c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.121875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.122479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.122764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.123036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.154290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.154822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.155103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.155375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.156035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:06.156336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.157738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.166070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 45/84] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:06.166839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.170003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.190897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.192376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.264051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.265140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd137eda-6326-403d-91d5-97297d2cb190 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.289920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 40/85] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:06.290732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.302806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.303704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.303991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.304266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.305764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.307223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.308841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 46/86] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:06.309726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.378022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.388490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f9a8d-f8dd-4bf9-82e1-dcc0ed89447b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.408020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dc766-3aa7-4ac0-a942-593bacf62f04 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.422173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.422850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.423137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.423413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.424054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.425573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.440510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.441111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.441427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.441702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.451543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 41/87] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:06 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:06.452360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.513325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.517838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.519347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.599947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd312964-a8c1-49bd-9259-aa3fe97fc140 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.601183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.638640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 47/88] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:06.639339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.648914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.649653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.649939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.651232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.652419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.653972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.655727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 42/89] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:06.660877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.758522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab2249-2041-487b-a57e-4c222852d25d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.788876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.794284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39e69-cccf-4c7d-a407-87a68365e56e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:06.798737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.799382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.799664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.799938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.801123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.802665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.804735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 48/90] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:06.810598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.837476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:06.838389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:06.838911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:06.839318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:06.840284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=474f14ac319744fba24161f36fcf9aa3) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:06.840782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:06.842447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:06.851006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 43/91] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/role_assignments?user.id=474f14ac319744fba24161f36fcf9aa3&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:06.851987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:06.866203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:06.969229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7a3d0-0d15-43ca-bca8-80ad8986170b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:07.010395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:07.011424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:07.011842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:07.012252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:07.013620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:07.015245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:07.017215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 49/92] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:07.018240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:07.083531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad60c6f-6146-437f-b2c6-b75b4d05747d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:07.144368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:07.144989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:07.145291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:07.145567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:07.150710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f987a0-8174-4eb3-965e-581e42e505c0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:07.167512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:07.201777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:07.202529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:07.202817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:07.203094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:07.203827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=6816207d6f6f4925a957b10d4c6e8140) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:07.204137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:07.204515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:07.205805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:07.213024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 50/93] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:07 2022] GET /identity/v3/role_assignments?user.id=6816207d6f6f4925a957b10d4c6e8140&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:07.214108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:07.314247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:07.316060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:07.375203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fd2ba-1e6e-49bd-8489-c12a5fcf1a72 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:07.438719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:07.439358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:07.439647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:07.439929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:07.440678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] RBAC: Adding query filter params (role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=6816207d6f6f4925a957b10d4c6e8140) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:07.440995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:07.442512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:07.456323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 51/94] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:07 2022] GET /identity/v3/role_assignments?user.id=6816207d6f6f4925a957b10d4c6e8140&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=807306af822540d7986dfb93d0b3f344 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:07.556613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 44/95] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:07.562207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:07.626862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:07.708696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6e7f3-0dab-4bf1-9dda-fe2dc3413d0d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:07.745402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42722e-8855-45fb-a507-e0d58ab1f7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:07.746055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42722e-8855-45fb-a507-e0d58ab1f7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:07.746341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42722e-8855-45fb-a507-e0d58ab1f7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:07.746623 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42722e-8855-45fb-a507-e0d58ab1f7e2 None admin] PATH_INFO: `/v3/projects/eb422529b07d437293dc9098a3501a33` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:07.747768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42722e-8855-45fb-a507-e0d58ab1f7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=eb422529b07d437293dc9098a3501a33)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:07.749232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42722e-8855-45fb-a507-e0d58ab1f7e2 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:07.751220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 52/96] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:07 2022] GET /identity/v3/projects/eb422529b07d437293dc9098a3501a33 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:07.756650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:07.826440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:07.876191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918f71e-dfbf-4386-8a66-843449e9b798 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:07.926188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a03a48-4d3c-4f8f-97fc-a37a50c0bf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:07.926809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a03a48-4d3c-4f8f-97fc-a37a50c0bf7c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:07.927097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a03a48-4d3c-4f8f-97fc-a37a50c0bf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:07.927373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a03a48-4d3c-4f8f-97fc-a37a50c0bf7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:07.928108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a03a48-4d3c-4f8f-97fc-a37a50c0bf7c None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=eb422529b07d437293dc9098a3501a33, user.id=6816207d6f6f4925a957b10d4c6e8140) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:07.928425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a03a48-4d3c-4f8f-97fc-a37a50c0bf7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:07.929930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a03a48-4d3c-4f8f-97fc-a37a50c0bf7c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:07.941371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 45/97] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:07 2022] GET /identity/v3/role_assignments?user.id=6816207d6f6f4925a957b10d4c6e8140&scope.project.id=eb422529b07d437293dc9098a3501a33&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:09.390452 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f13bb-3310-4d80-9c2b-4abb54359fcb None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:09.390985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f13bb-3310-4d80-9c2b-4abb54359fcb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:09.390985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f13bb-3310-4d80-9c2b-4abb54359fcb None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:09.392625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 53/98] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:09.398312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101624a-f4e7-43a3-b8f2-ee0ac5f78db0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:09.398698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101624a-f4e7-43a3-b8f2-ee0ac5f78db0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:09.398873 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101624a-f4e7-43a3-b8f2-ee0ac5f78db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:09.426409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d101624a-f4e7-43a3-b8f2-ee0ac5f78db0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:09.427623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d101624a-f4e7-43a3-b8f2-ee0ac5f78db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:09.526692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 46/99] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:09.554097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944a7d7-5fda-42bd-810d-a8f279cf9d77 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:09.554383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944a7d7-5fda-42bd-810d-a8f279cf9d77 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:09.554659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944a7d7-5fda-42bd-810d-a8f279cf9d77 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:09.556584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 54/100] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:09.562071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81325197-0fa6-4c3f-8df6-a5745b7f2897 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:09.562358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81325197-0fa6-4c3f-8df6-a5745b7f2897 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:09.562633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81325197-0fa6-4c3f-8df6-a5745b7f2897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:09.569419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944a7d7-5fda-42bd-810d-a8f279cf9d77 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:09.598499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-81325197-0fa6-4c3f-8df6-a5745b7f2897 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:09.604021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-81325197-0fa6-4c3f-8df6-a5745b7f2897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:09.705559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:09.736686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 47/101] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:09.742091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201311e-9967-408e-873c-b53c6ce68443 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:09.742378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201311e-9967-408e-873c-b53c6ce68443 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:09.742655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201311e-9967-408e-873c-b53c6ce68443 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:09.745204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 48/102] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:09.750729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b9f35-b1a2-4214-a0e0-ee0ecf4d1e38 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:09.751739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b9f35-b1a2-4214-a0e0-ee0ecf4d1e38 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:09.752018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b9f35-b1a2-4214-a0e0-ee0ecf4d1e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:09.775339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944a7d7-5fda-42bd-810d-a8f279cf9d77 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:09.784222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:09.794195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-fb9b9f35-b1a2-4214-a0e0-ee0ecf4d1e38 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:09.795412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-fb9b9f35-b1a2-4214-a0e0-ee0ecf4d1e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:09.840559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:09.841533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:09.841866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:09.842146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:09.843669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:09.845136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:09.846913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 55/103] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:09.851766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:09.923888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 49/104] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:09.925048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b9f35-b1a2-4214-a0e0-ee0ecf4d1e38 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.031731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce577dd-6b4c-48e4-9306-e1f5654ddff1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.075073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.075738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.076027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.076308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.090709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:10.095808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:10.096644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:10.097413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 56/105] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:10.098047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.118148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b9f35-b1a2-4214-a0e0-ee0ecf4d1e38 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.132748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:10.136832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:10.161598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9b822-09d5-4c94-a403-79b530fbf090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.162226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9b822-09d5-4c94-a403-79b530fbf090 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.162513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9b822-09d5-4c94-a403-79b530fbf090 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.162791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9b822-09d5-4c94-a403-79b530fbf090 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.163458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9b822-09d5-4c94-a403-79b530fbf090 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:10.164985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9b822-09d5-4c94-a403-79b530fbf090 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:10.197295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 50/106] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:18:09 2022] POST /identity/v3/projects => generated 302 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:10.197295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28b9b-dc46-49d1-828a-206ec25273f4 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.197295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28b9b-dc46-49d1-828a-206ec25273f4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.197295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28b9b-dc46-49d1-828a-206ec25273f4 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.197295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 51/107] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:10.197295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f2be-aeca-45cc-aafd-3f6bb7044862 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.197295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f2be-aeca-45cc-aafd-3f6bb7044862 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.197821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26f2be-aeca-45cc-aafd-3f6bb7044862 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.197821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 52/108] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:10.197821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f2be-aeca-45cc-aafd-3f6bb7044862 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.238215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d7d0b-4992-4277-8fd1-0d33d9700f05 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.283704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05018d-a026-4996-a421-8fd7735c3af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.284385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05018d-a026-4996-a421-8fd7735c3af4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.284806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05018d-a026-4996-a421-8fd7735c3af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.285140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05018d-a026-4996-a421-8fd7735c3af4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.298347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05018d-a026-4996-a421-8fd7735c3af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-1c05018d-a026-4996-a421-8fd7735c3af4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:10.304015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:10.304897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:10.305725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 57/109] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:10.309290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872becb3-edfe-4f30-8c22-f831a699f3b8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.309290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872becb3-edfe-4f30-8c22-f831a699f3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.309290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872becb3-edfe-4f30-8c22-f831a699f3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.328958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-872becb3-edfe-4f30-8c22-f831a699f3b8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:10.330284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-872becb3-edfe-4f30-8c22-f831a699f3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:10.346504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26f2be-aeca-45cc-aafd-3f6bb7044862 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.403922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.404623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.404623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.404623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.405708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:10.405708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:10.407089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:10.416414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 58/110] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:10.418467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a4349-b3f2-4d43-86c1-8cf2efbbcffc None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.418948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a4349-b3f2-4d43-86c1-8cf2efbbcffc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.419436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 53/111] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:10.419850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a4349-b3f2-4d43-86c1-8cf2efbbcffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.424445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.440417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-674a4349-b3f2-4d43-86c1-8cf2efbbcffc None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:10.442044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-674a4349-b3f2-4d43-86c1-8cf2efbbcffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:10.460024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:10.468752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:10.542924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31217f-3b4d-46f3-b4c9-28f04f35fa39 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.543575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 59/112] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:10.549575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a4349-b3f2-4d43-86c1-8cf2efbbcffc None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.601480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.602336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.602445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.602683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.603747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:10.603927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:10.605296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:10.614868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 54/113] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:10.615532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.705840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a4349-b3f2-4d43-86c1-8cf2efbbcffc None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.718558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:10.729334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b4978-e1e7-431a-9d44-315ca4431435 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.759801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.760743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.761207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.761677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] PATH_INFO: `/v3/users/4eedef18683d42f28aee1079cd0d8aaa` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.766588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.767293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.767642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.767984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.779286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:10.780111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] RBAC: Authorizing `identity:get_user(user_id=4eedef18683d42f28aee1079cd0d8aaa)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:10.781842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:10.783707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 60/114] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/users/4eedef18683d42f28aee1079cd0d8aaa => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:10.784645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:10.785027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:10.785921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:10.786710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 55/115] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:10.787335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.808978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:10.907624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044679be-942e-47df-bbf2-5397ed094d26 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.918129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f3dd8-b633-4005-872d-a2ed2033f932 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:10.944821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.945657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.946266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.946381 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.957814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:10.962664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:10.963711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:10.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:10.965720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:10.966425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 61/116] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:10.966667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:10.967104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:10.967558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:10.967715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] PATH_INFO: `/v3/users/474f14ac319744fba24161f36fcf9aa3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:10.969514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] RBAC: Authorizing `identity:get_user(user_id=474f14ac319744fba24161f36fcf9aa3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:10.971137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:10.974373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 56/117] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/users/474f14ac319744fba24161f36fcf9aa3 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:10.975337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.012048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.052500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.085411 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fadfd9-e916-4354-8f64-0501b1a3cd05 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.097619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79fff3-4013-48f1-ba51-937622dd948f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.123770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.124381 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.124662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.124936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.134043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.134658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.134939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.135211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.135869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:11.136172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.137586 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.145666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 57/118] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:11.146255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.195989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.197992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.234727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.257725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83347-5e0f-4a46-9e96-1d7bde2a6e3e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.266504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.297825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366be3a2-1827-4af4-b63e-64d95aa73b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.298436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366be3a2-1827-4af4-b63e-64d95aa73b04 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.298722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366be3a2-1827-4af4-b63e-64d95aa73b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.299003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366be3a2-1827-4af4-b63e-64d95aa73b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.299664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366be3a2-1827-4af4-b63e-64d95aa73b04 None admin] RBAC: Adding query filter params (name=member) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:11.299975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366be3a2-1827-4af4-b63e-64d95aa73b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.301404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366be3a2-1827-4af4-b63e-64d95aa73b04 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.308942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 62/119] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:11.308942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.313387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 58/120] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:11.314996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e764c-adaa-4d09-9849-54d55e103d03 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.315807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e764c-adaa-4d09-9849-54d55e103d03 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.316098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e764c-adaa-4d09-9849-54d55e103d03 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.317974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 59/121] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:11.318615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e764c-adaa-4d09-9849-54d55e103d03 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.416469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080ca29-47f3-412f-bd37-7e2c9b1680ba None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.429148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e764c-adaa-4d09-9849-54d55e103d03 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.454613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.455262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.455543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.455819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.456958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.458476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.460118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 63/122] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:11.460869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.478362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.479727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.480221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.480958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.482309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.484403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.486642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 60/123] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:11.491451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.506677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.508184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.572168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0334cc8-67b0-4b9c-b9f7-ffd1cb0f1594 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.604498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413baabd-6370-489e-babb-4c94782fb677 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.614257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac31fa2-acfc-4076-a9cb-5b8d64a2449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.614900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac31fa2-acfc-4076-a9cb-5b8d64a2449a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.615311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac31fa2-acfc-4076-a9cb-5b8d64a2449a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.615573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac31fa2-acfc-4076-a9cb-5b8d64a2449a None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.616700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac31fa2-acfc-4076-a9cb-5b8d64a2449a None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.618414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac31fa2-acfc-4076-a9cb-5b8d64a2449a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.625017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 64/124] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:11.625017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3fac5-b723-499b-bd75-65c22741205f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.625017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3fac5-b723-499b-bd75-65c22741205f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.625017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3fac5-b723-499b-bd75-65c22741205f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.640432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-00b3fac5-b723-499b-bd75-65c22741205f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:11.641645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-00b3fac5-b723-499b-bd75-65c22741205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:11.643879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.644509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.644785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.645058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.646195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.647636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.649828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 61/125] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:11.650556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.715594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 65/126] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:11.716688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3fac5-b723-499b-bd75-65c22741205f None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.742611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.763471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:11.765869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee804ab7-ed62-4f0b-9a76-17360285b089 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.799852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.800509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.800788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.801064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/474f14ac319744fba24161f36fcf9aa3/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.803001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d75a1525d11453faac47809e5dc2f01, user_id=474f14ac319744fba24161f36fcf9aa3, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.804462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.818149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 62/127] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:11 2022] PUT /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/474f14ac319744fba24161f36fcf9aa3/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:11.818958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.854864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3fac5-b723-499b-bd75-65c22741205f None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:11.905868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:11.906521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:11.906805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:11.907133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:11.908407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:11.909960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:11.911858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 66/128] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:11.911858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:11.977692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9261800-c187-44b0-86d0-34b95821bad7 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:12.016737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:12.017681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.018138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.018618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.019530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] RBAC: Adding query filter params (role.id=97e9dc3569a94caeb432886a8e5e708c, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=4eedef18683d42f28aee1079cd0d8aaa) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:12.020034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:12.021617 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:12.029087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 63/129] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/role_assignments?user.id=4eedef18683d42f28aee1079cd0d8aaa&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=97e9dc3569a94caeb432886a8e5e708c => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:12.029757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:12.034394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:12.076195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ae98f-74b7-48c7-bde3-778e84099dc2 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:12.083535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:12.163922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:12.164653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.165460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.165739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.166990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:12.167744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deee7da-52c5-4eb8-af8d-081a490a92a6 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:12.168473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:12.175007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 67/130] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:11 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:12.175007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:12.231999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:12.232725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.233105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.233573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.250016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:12.256964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:12.258021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:12.259387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:12.261227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 64/131] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:12.261227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:12.337641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708996b-642b-4297-817d-4dcaa788ca8e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:12.380241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:12.389687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:12.390357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.390642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.390924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/6816207d6f6f4925a957b10d4c6e8140/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.393123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] RBAC: Authorizing `identity:create_grant(project_id=6d75a1525d11453faac47809e5dc2f01, user_id=6816207d6f6f4925a957b10d4c6e8140, role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:12.394707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:12.413309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 68/132] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:12 2022] PUT /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/6816207d6f6f4925a957b10d4c6e8140/roles/807306af822540d7986dfb93d0b3f344 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:12.417639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:12.437182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5ca09-3d4f-47ed-9a87-cd0727ef2099 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:12.491737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:12.508239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4182-f8fd-44e9-8656-f3e56e139d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:12.509884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4182-f8fd-44e9-8656-f3e56e139d92 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.510371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4182-f8fd-44e9-8656-f3e56e139d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.510846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4182-f8fd-44e9-8656-f3e56e139d92 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/6816207d6f6f4925a957b10d4c6e8140/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.515415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4182-f8fd-44e9-8656-f3e56e139d92 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d75a1525d11453faac47809e5dc2f01, user_id=6816207d6f6f4925a957b10d4c6e8140, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:12.517359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4182-f8fd-44e9-8656-f3e56e139d92 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:12.543330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 65/133] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:12 2022] PUT /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/6816207d6f6f4925a957b10d4c6e8140/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:12.546278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ae05-a50e-4916-ae9d-43462335ce6e None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.546908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ae05-a50e-4916-ae9d-43462335ce6e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.547421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268ae05-a50e-4916-ae9d-43462335ce6e None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.551089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 66/134] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:12.559043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18499f74-7f16-4f90-87b8-99fc9264264b None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.559682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18499f74-7f16-4f90-87b8-99fc9264264b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.560430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18499f74-7f16-4f90-87b8-99fc9264264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.609335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-18499f74-7f16-4f90-87b8-99fc9264264b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:12.610933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-18499f74-7f16-4f90-87b8-99fc9264264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:12.652125 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03351542-4dca-41aa-ac28-eb36e1cf3efc None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:12.698073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:12.699090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.699761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.700236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.701126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:12.701659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:12.703978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:12.715128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 69/135] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:12.722295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:12.746876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 67/136] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:12.827492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18499f74-7f16-4f90-87b8-99fc9264264b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:12.843477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:12.850010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372f237-d903-4f78-92aa-d70aeb384c1b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:12.888460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:12.889106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:12.889429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:12.889711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:12.890883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:12.892357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:12.894008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 70/137] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:12 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:12.900058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:12.920985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:12.993557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18499f74-7f16-4f90-87b8-99fc9264264b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:13.009425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943036-7690-4c8c-abae-05230edbc52a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:13.047822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:13.048482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:13.048771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:13.049040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] PATH_INFO: `/v3/projects/eb422529b07d437293dc9098a3501a33` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:13.050512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] RBAC: Authorizing `identity:get_project(project_id=eb422529b07d437293dc9098a3501a33)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:13.051999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:13.053685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 71/138] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:12 2022] GET /identity/v3/projects/eb422529b07d437293dc9098a3501a33 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:13.060734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:13.097685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:13.107006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:13.107969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:13.108256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:13.108559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:13.112867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:13.114394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:13.116198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 68/139] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:13.124951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:13.177723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38766d-8ae1-4148-b282-84b643ff119e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:13.219707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bbda9-747d-450d-a196-881417a1c013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:13.220546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bbda9-747d-450d-a196-881417a1c013 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:13.220930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bbda9-747d-450d-a196-881417a1c013 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:13.221475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bbda9-747d-450d-a196-881417a1c013 None admin] PATH_INFO: `/v3/projects/eb422529b07d437293dc9098a3501a33/users/6816207d6f6f4925a957b10d4c6e8140/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:13.223983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bbda9-747d-450d-a196-881417a1c013 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb422529b07d437293dc9098a3501a33, user_id=6816207d6f6f4925a957b10d4c6e8140, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:13.226175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bbda9-747d-450d-a196-881417a1c013 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:13.244587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 72/140] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:13 2022] PUT /identity/v3/projects/eb422529b07d437293dc9098a3501a33/users/6816207d6f6f4925a957b10d4c6e8140/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:18:13.318726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4ef39-0442-4bb6-9a77-156420b63f07 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:13.352643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:13.368419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbff1fd-ccbf-49f6-b057-b1f16c9e10bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:13.369024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff1fd-ccbf-49f6-b057-b1f16c9e10bb None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:13.369331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff1fd-ccbf-49f6-b057-b1f16c9e10bb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:13.369610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff1fd-ccbf-49f6-b057-b1f16c9e10bb None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:13.370324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbff1fd-ccbf-49f6-b057-b1f16c9e10bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:13.371922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbff1fd-ccbf-49f6-b057-b1f16c9e10bb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:13.398554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 69/141] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:13 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:14.295912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381d5fc-5f70-4dc0-ac35-7b3c3b4f2681 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:14.296908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381d5fc-5f70-4dc0-ac35-7b3c3b4f2681 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:14.297418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381d5fc-5f70-4dc0-ac35-7b3c3b4f2681 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:14.299005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 73/142] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:14.304325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c2e55-a4c7-43c3-a626-d1128cda1351 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:14.304740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c2e55-a4c7-43c3-a626-d1128cda1351 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:14.304913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c2e55-a4c7-43c3-a626-d1128cda1351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:14.330461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3d9c2e55-a4c7-43c3-a626-d1128cda1351 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:14.331749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3d9c2e55-a4c7-43c3-a626-d1128cda1351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:14.409912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 70/143] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:14.452781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381d5fc-5f70-4dc0-ac35-7b3c3b4f2681 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:14.552181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:14.567730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381d5fc-5f70-4dc0-ac35-7b3c3b4f2681 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:14.611508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:14.612146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:14.612421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:14.612691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:14.629174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:14.634031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:14.634836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:14.635590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 74/144] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:14.640956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c2e55-a4c7-43c3-a626-d1128cda1351 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:14.760947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c2e55-a4c7-43c3-a626-d1128cda1351 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:14.771027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:14.800630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:14.801246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:14.801546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:14.801816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:14.802538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:14.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:14.804275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:14.816625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 71/145] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:14.822162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:14.936136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15531e5e-1a57-4c70-8f66-9739d477a83f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:14.974643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2fe99-0ff9-40c3-bceb-c8288491bebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:14.975272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2fe99-0ff9-40c3-bceb-c8288491bebe None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:14.975547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2fe99-0ff9-40c3-bceb-c8288491bebe None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:14.975814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2fe99-0ff9-40c3-bceb-c8288491bebe None admin] PATH_INFO: `/v3/users/474f14ac319744fba24161f36fcf9aa3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:14.976974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2fe99-0ff9-40c3-bceb-c8288491bebe None admin] RBAC: Authorizing `identity:get_user(user_id=474f14ac319744fba24161f36fcf9aa3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:14.979233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2fe99-0ff9-40c3-bceb-c8288491bebe None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:14.980984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 75/146] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:14 2022] GET /identity/v3/users/474f14ac319744fba24161f36fcf9aa3 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:14.986412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:15.096419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88a68-eeaa-4718-a1b2-187e9d13e162 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:15.134520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:15.135180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.135463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.135735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.182501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:15.208711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:15.210203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:15.240559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384f29e-5daf-4595-b9ea-ca955af9daaa None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.240852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384f29e-5daf-4595-b9ea-ca955af9daaa None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.241100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384f29e-5daf-4595-b9ea-ca955af9daaa None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.242666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 76/147] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:15.247701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e44f2f-9d71-46cb-8056-26ad5d13450a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.248050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e44f2f-9d71-46cb-8056-26ad5d13450a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.248325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e44f2f-9d71-46cb-8056-26ad5d13450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.282905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c3e44f2f-9d71-46cb-8056-26ad5d13450a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:15.288669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c3e44f2f-9d71-46cb-8056-26ad5d13450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:15.316948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:15.440131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 77/148] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:15.448716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 72/149] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 12:18:15.454149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e44f2f-9d71-46cb-8056-26ad5d13450a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:15.490682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:15.589354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e44f2f-9d71-46cb-8056-26ad5d13450a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:15.635991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:15.646076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d3c83-78ff-4c3e-a9de-90b3049334b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:15.646737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3c83-78ff-4c3e-a9de-90b3049334b1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.647070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3c83-78ff-4c3e-a9de-90b3049334b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.647343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3c83-78ff-4c3e-a9de-90b3049334b1 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.648554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d3c83-78ff-4c3e-a9de-90b3049334b1 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:15.650315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d3c83-78ff-4c3e-a9de-90b3049334b1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:15.652298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 78/150] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:15 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:15.653566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56be7-c06f-471d-a0fc-c3e22648f714 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.653851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56be7-c06f-471d-a0fc-c3e22648f714 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.654126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56be7-c06f-471d-a0fc-c3e22648f714 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.655344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 79/151] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:15.660227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56be7-c06f-471d-a0fc-c3e22648f714 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:15.685788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda52ca-011b-430c-8a6e-8a38b7016b66 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:15.703211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:15.730808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8b602-0e38-446a-ae49-cb77c0802086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:15.731680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8b602-0e38-446a-ae49-cb77c0802086 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.732099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8b602-0e38-446a-ae49-cb77c0802086 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.732507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8b602-0e38-446a-ae49-cb77c0802086 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.744210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8b602-0e38-446a-ae49-cb77c0802086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-91c8b602-0e38-446a-ae49-cb77c0802086 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:15.749819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:15.750644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:15.751387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:15.754151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 73/152] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:15.755516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ff936-92ae-4d52-a7da-bc469c1b2d96 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.755989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ff936-92ae-4d52-a7da-bc469c1b2d96 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.756449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ff936-92ae-4d52-a7da-bc469c1b2d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.775677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b35ff936-92ae-4d52-a7da-bc469c1b2d96 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:15.777163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b35ff936-92ae-4d52-a7da-bc469c1b2d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:15.846518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 74/153] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:15.847733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ff936-92ae-4d52-a7da-bc469c1b2d96 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:15.907853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56be7-c06f-471d-a0fc-c3e22648f714 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:15.947741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:15.948587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:15.950292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:15.950292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:15.950680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=474f14ac319744fba24161f36fcf9aa3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:15.950992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:15.952480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:15.958711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 80/154] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:15 2022] GET /identity/v3/role_assignments?user.id=474f14ac319744fba24161f36fcf9aa3&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:15.959447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:15.986466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:16.074438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293997-70df-4f6e-929e-be5787dc9949 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:16.077572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:16.121682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c01a2-e316-4eec-bc79-f2c0d9b71a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:16.122348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c01a2-e316-4eec-bc79-f2c0d9b71a3d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:16.122630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c01a2-e316-4eec-bc79-f2c0d9b71a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:16.122903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c01a2-e316-4eec-bc79-f2c0d9b71a3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:16.138802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c01a2-e316-4eec-bc79-f2c0d9b71a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-730c01a2-e316-4eec-bc79-f2c0d9b71a3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:16.144821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:16.145828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:16.147116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 81/155] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:16.152738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d3399-2422-476e-b6fa-52881e0e4dcd None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:16.153044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d3399-2422-476e-b6fa-52881e0e4dcd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:16.153328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d3399-2422-476e-b6fa-52881e0e4dcd None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:16.154663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 82/156] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:16.155524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d3399-2422-476e-b6fa-52881e0e4dcd None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:16.169425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ff936-92ae-4d52-a7da-bc469c1b2d96 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:16.237145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4e370-e3c9-4207-862f-39d565fd9877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:16.237753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4e370-e3c9-4207-862f-39d565fd9877 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:16.238041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4e370-e3c9-4207-862f-39d565fd9877 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:16.238321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4e370-e3c9-4207-862f-39d565fd9877 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:16.239021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4e370-e3c9-4207-862f-39d565fd9877 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:16.239330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4e370-e3c9-4207-862f-39d565fd9877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:16.245299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4e370-e3c9-4207-862f-39d565fd9877 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:16.254020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 75/157] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:16.255333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd14c-c135-4cc7-afa7-6acfabff8ba6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:16.255621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd14c-c135-4cc7-afa7-6acfabff8ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:16.256103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd14c-c135-4cc7-afa7-6acfabff8ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:16.294610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9c0bd14c-c135-4cc7-afa7-6acfabff8ba6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:16.295841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9c0bd14c-c135-4cc7-afa7-6acfabff8ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:16.402844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d3399-2422-476e-b6fa-52881e0e4dcd None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:16.457603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 76/158] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:16.458461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd14c-c135-4cc7-afa7-6acfabff8ba6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:16.472864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:16.473550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:16.473826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:16.474144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:16.474814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:16.475162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:16.476611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:16.494373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 83/159] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:16.499399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:16.521629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:16.561674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:16.626627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd14c-c135-4cc7-afa7-6acfabff8ba6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:16.700587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:16.701244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:16.701637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:16.701915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] PATH_INFO: `/v3/users/4eedef18683d42f28aee1079cd0d8aaa` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:16.704887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] RBAC: Authorizing `identity:get_user(user_id=4eedef18683d42f28aee1079cd0d8aaa)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:16.706405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:16.708159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 77/160] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:16 2022] GET /identity/v3/users/4eedef18683d42f28aee1079cd0d8aaa => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:16.708943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:16.725195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972433-5abf-42fb-aa7b-275d84409fe9 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:16.810334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:16.810908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:16.811187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:16.811460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:16.812622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:16.814107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:16.815686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 84/161] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:16 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:16.817536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:16.935309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b8ac5-8419-4b01-8f6b-556147d6564d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:16.973653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:17.009647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:17.017978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:17.018636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.018927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.019208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.041198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:17.046392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:17.047297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:17.048110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 78/162] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:17.048795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:17.064092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698cc1-546e-42af-93a4-1b075e69a104 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:17.125440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117185aa-40df-4ed2-98a7-643d90e64178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:17.126083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117185aa-40df-4ed2-98a7-643d90e64178 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.126359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117185aa-40df-4ed2-98a7-643d90e64178 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.126631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117185aa-40df-4ed2-98a7-643d90e64178 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.134594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117185aa-40df-4ed2-98a7-643d90e64178 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:17.136071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117185aa-40df-4ed2-98a7-643d90e64178 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:17.138039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 85/163] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:16 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:17.139024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fca5e-fc0b-4094-88ab-32d4c3879672 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.139302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fca5e-fc0b-4094-88ab-32d4c3879672 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.139573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fca5e-fc0b-4094-88ab-32d4c3879672 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.140757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 86/164] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:17.141464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fca5e-fc0b-4094-88ab-32d4c3879672 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:17.183308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b630f-a5a6-4b1d-9ce5-e16349d4cadb None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:17.249510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03526-f4e1-4b64-9456-ab4cc0cebca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:17.250127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03526-f4e1-4b64-9456-ab4cc0cebca7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.250410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03526-f4e1-4b64-9456-ab4cc0cebca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.250684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03526-f4e1-4b64-9456-ab4cc0cebca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.331880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fca5e-fc0b-4094-88ab-32d4c3879672 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:17.361645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:17.387122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:17.387885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.388173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.388450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.389127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:17.389462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:17.390945 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03526-f4e1-4b64-9456-ab4cc0cebca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:17.391061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:17.392517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03526-f4e1-4b64-9456-ab4cc0cebca7 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:17.397345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:17.410370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 87/165] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:17.414707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:17.538317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 79/166] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:17.540652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a261c0-d352-4981-9650-4c31da8464de None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.541203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a261c0-d352-4981-9650-4c31da8464de None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.541503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a261c0-d352-4981-9650-4c31da8464de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.555167 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965157-dcab-443e-b1d3-fdb647b37fff None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:17.562264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-29a261c0-d352-4981-9650-4c31da8464de None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:17.563438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-29a261c0-d352-4981-9650-4c31da8464de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:17.595064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:17.595064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.595064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.595064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/4eedef18683d42f28aee1079cd0d8aaa/roles/97e9dc3569a94caeb432886a8e5e708c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.596809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d75a1525d11453faac47809e5dc2f01, user_id=4eedef18683d42f28aee1079cd0d8aaa, role_id=97e9dc3569a94caeb432886a8e5e708c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:17.606539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:17.616776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 88/167] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:17 2022] PUT /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01/users/4eedef18683d42f28aee1079cd0d8aaa/roles/97e9dc3569a94caeb432886a8e5e708c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:17.618106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:17.652434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 80/168] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:17.654039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea7e8-c613-46fe-bc16-932d918ba96f None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.654322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea7e8-c613-46fe-bc16-932d918ba96f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.654595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea7e8-c613-46fe-bc16-932d918ba96f None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.655893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 81/169] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:17.661213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea7e8-c613-46fe-bc16-932d918ba96f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:17.682432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:17.702249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:17.780254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2a3a-ac00-48ef-a80e-b39ca05c2600 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:17.815630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68930b7-73e4-47e4-b168-42dc52004a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:17.816630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68930b7-73e4-47e4-b168-42dc52004a02 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.817053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68930b7-73e4-47e4-b168-42dc52004a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.817501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68930b7-73e4-47e4-b168-42dc52004a02 None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.818865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68930b7-73e4-47e4-b168-42dc52004a02 None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:17.821416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68930b7-73e4-47e4-b168-42dc52004a02 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:17.827014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 89/170] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:17 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:17.828525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946eeb7-4c24-41e3-bc4d-b37227b641f4 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.829000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946eeb7-4c24-41e3-bc4d-b37227b641f4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.829494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946eeb7-4c24-41e3-bc4d-b37227b641f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.831507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea7e8-c613-46fe-bc16-932d918ba96f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:17.850701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8946eeb7-4c24-41e3-bc4d-b37227b641f4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:17.852240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8946eeb7-4c24-41e3-bc4d-b37227b641f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:17.876559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:17.877460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:17.877896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:17.878294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:17.880054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:17.882126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:17.885576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 82/171] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:17 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:17.886508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:17.937214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 90/172] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:17.940804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946eeb7-4c24-41e3-bc4d-b37227b641f4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.016322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.026026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.048878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceedbc-3e81-4060-92ab-4103a6ba2f46 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.069864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946eeb7-4c24-41e3-bc4d-b37227b641f4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.094690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.095605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.095886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.096404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.108184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c0d0e-f10f-48e3-8191-361d7e46c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.108752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0d0e-f10f-48e3-8191-361d7e46c958 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.109024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0d0e-f10f-48e3-8191-361d7e46c958 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.109306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0d0e-f10f-48e3-8191-361d7e46c958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.110093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:18.116776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:18.118367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:18.119143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:18.124923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 83/173] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:18.125985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.207510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c0d0e-f10f-48e3-8191-361d7e46c958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:18.209976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c0d0e-f10f-48e3-8191-361d7e46c958 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:18.245097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53228a3-8a14-4543-82a7-c09e8ca8bcbd None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.282966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.283591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.283862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.284132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.284866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] RBAC: Adding query filter params (role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=6816207d6f6f4925a957b10d4c6e8140) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:18.285179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:18.286668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:18.287561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.293399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 84/174] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/role_assignments?user.id=6816207d6f6f4925a957b10d4c6e8140&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=807306af822540d7986dfb93d0b3f344 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:18.294118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.296687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.347912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 91/175] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:18.349748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f89f2-a283-472b-aa0b-fd6df08b1cd2 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.349909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f89f2-a283-472b-aa0b-fd6df08b1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.350212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f89f2-a283-472b-aa0b-fd6df08b1cd2 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.356594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 92/176] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:18.356594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f89f2-a283-472b-aa0b-fd6df08b1cd2 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.440989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5ea62-9ef7-41bf-9152-6e4bfa65c28e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.480217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878bf1a-b7c4-4460-980e-e275d2d8df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.480962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878bf1a-b7c4-4460-980e-e275d2d8df33 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.481331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878bf1a-b7c4-4460-980e-e275d2d8df33 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.481676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878bf1a-b7c4-4460-980e-e275d2d8df33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.484930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f89f2-a283-472b-aa0b-fd6df08b1cd2 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.495251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878bf1a-b7c4-4460-980e-e275d2d8df33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-6878bf1a-b7c4-4460-980e-e275d2d8df33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:18.500562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:18.501579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:18.502387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 85/177] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:18.503364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2394-40c9-4bcf-9fb1-e5d92cc17c2c None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.503538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2394-40c9-4bcf-9fb1-e5d92cc17c2c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.503801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2394-40c9-4bcf-9fb1-e5d92cc17c2c None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.505155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 86/178] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:18.510783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd2394-40c9-4bcf-9fb1-e5d92cc17c2c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.522431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251edea-d5b4-467e-98de-43c816d31224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.523069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251edea-d5b4-467e-98de-43c816d31224 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.523322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251edea-d5b4-467e-98de-43c816d31224 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.523595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251edea-d5b4-467e-98de-43c816d31224 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.524308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251edea-d5b4-467e-98de-43c816d31224 None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:18.524609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251edea-d5b4-467e-98de-43c816d31224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:18.526038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251edea-d5b4-467e-98de-43c816d31224 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:18.536529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 93/179] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:18.537804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff0d2-ab28-44af-b4fa-9ded776bd9df None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.538087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff0d2-ab28-44af-b4fa-9ded776bd9df None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.538354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff0d2-ab28-44af-b4fa-9ded776bd9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.561025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-dd8ff0d2-ab28-44af-b4fa-9ded776bd9df None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:18.562271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-dd8ff0d2-ab28-44af-b4fa-9ded776bd9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:18.584455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.591249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.637045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd2394-40c9-4bcf-9fb1-e5d92cc17c2c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.643213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 94/180] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:18.644858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56595f73-e1e4-44fe-a987-cf9b2f0430b8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.645144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56595f73-e1e4-44fe-a987-cf9b2f0430b8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.645453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56595f73-e1e4-44fe-a987-cf9b2f0430b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.669861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-56595f73-e1e4-44fe-a987-cf9b2f0430b8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:18.671037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-56595f73-e1e4-44fe-a987-cf9b2f0430b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:18.676925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.677633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.677916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.678182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.679772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:18.681679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:18.684060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 87/181] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:18.688827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.738138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 95/182] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:18.738923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56595f73-e1e4-44fe-a987-cf9b2f0430b8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.806088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b11645-2423-4d60-af3f-5ed33a843df9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.829911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.838832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:18.844992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.845657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.845902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.846169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.846823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:18.847120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:18.848732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:18.852917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56595f73-e1e4-44fe-a987-cf9b2f0430b8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:18.859736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 88/183] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:18.860702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.892764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:18.893429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:18.893704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:18.893973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] PATH_INFO: `/v3/users/6816207d6f6f4925a957b10d4c6e8140` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:18.895138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=6816207d6f6f4925a957b10d4c6e8140)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:18.896650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:18.898253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 96/184] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/users/6816207d6f6f4925a957b10d4c6e8140 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:18.899574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:18.978147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f94635-c821-4161-93f4-27215b20d958 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.016574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf26bd-fbd1-475f-8bdd-768be362e2d9 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.019820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.020522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.020809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.021081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.034867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:19.041968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:19.042826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:19.043614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:19.045581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 89/185] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:19.046670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.058564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.059181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.059457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.060035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.060790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=6816207d6f6f4925a957b10d4c6e8140) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:19.061097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.062557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.068568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 97/186] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:18 2022] GET /identity/v3/role_assignments?user.id=6816207d6f6f4925a957b10d4c6e8140&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:19.077612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.094095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.130283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.193749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9aa49-5e8d-48a3-80c5-943f182a25ee None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.195056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d4789-e2e2-4617-9456-1ce5e1a62de5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.236207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.236755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.237034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.237370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.238523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.240114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.241932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 90/187] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:19.242661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.242900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.243845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.244340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.244840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] PATH_INFO: `/v3/users/c8c7c98cffd340a78515170c150fc447` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.259895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] RBAC: Authorizing `identity:get_user(user_id=c8c7c98cffd340a78515170c150fc447)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.262134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.263821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 98/188] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/users/c8c7c98cffd340a78515170c150fc447 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:19.264592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.350118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.360058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ee81-50e3-405a-95f2-d8cd158d269a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.382175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.390388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c40621-1808-46f2-8059-7aab50342e80 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.398041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.398653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.398971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.399267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.423518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.424127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.424401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.424669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.425358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:19.425660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.427036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.435418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 99/189] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:19.436538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.477171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.479058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.535536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc1f96-c7c8-4401-9d2e-b84c8a68bfe1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.564635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.570528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.571703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.571981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.572246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.572888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.574379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.597121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 91/190] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:19.597121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.602992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 100/191] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:19 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:19.604243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.637609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.734197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fe6b0-ec58-4ca9-b69f-27ed09b3bf22 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.752560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab73a8-decb-4431-9532-66e25580a652 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.773384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.774389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.774840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.775279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] PATH_INFO: `/v3/users/c8c7c98cffd340a78515170c150fc447` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.776677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] RBAC: Authorizing `identity:get_user(user_id=c8c7c98cffd340a78515170c150fc447)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.778237 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.779918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 101/192] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/users/c8c7c98cffd340a78515170c150fc447 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:19.780829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:19.803934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3661f8-8808-4a47-b347-f153046e43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.804938 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3661f8-8808-4a47-b347-f153046e43dd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.805981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3661f8-8808-4a47-b347-f153046e43dd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.806260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3661f8-8808-4a47-b347-f153046e43dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.848682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.890121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3661f8-8808-4a47-b347-f153046e43dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.891886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3661f8-8808-4a47-b347-f153046e43dd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.907668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b06f2-4218-4357-9047-7b95c8258a74 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:19.920169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:19.949840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:19.950539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:19.950828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:19.951107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] PATH_INFO: `/v3/projects/eb422529b07d437293dc9098a3501a33` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:19.952264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] RBAC: Authorizing `identity:get_project(project_id=eb422529b07d437293dc9098a3501a33)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:19.953781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:19.955599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 102/193] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/projects/eb422529b07d437293dc9098a3501a33 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:19.956205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:20.018629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 92/194] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:20.022747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565750f-184c-4d49-b3aa-ef76b7117bca None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.022747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565750f-184c-4d49-b3aa-ef76b7117bca None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.022747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565750f-184c-4d49-b3aa-ef76b7117bca None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.022747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 93/195] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:20.022747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565750f-184c-4d49-b3aa-ef76b7117bca None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:20.082240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf0e7b-782e-4a63-9716-84bce63e7b9a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:20.107405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:20.120556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:20.121191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.121495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.121762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.133653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565750f-184c-4d49-b3aa-ef76b7117bca None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:20.170215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:20.170829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.171107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.171379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.172102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=eb422529b07d437293dc9098a3501a33, user.id=6816207d6f6f4925a957b10d4c6e8140) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:20.172463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:20.173964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:20.180169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 94/196] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:20 2022] GET /identity/v3/role_assignments?user.id=6816207d6f6f4925a957b10d4c6e8140&scope.project.id=eb422529b07d437293dc9098a3501a33&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:20.180984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:20.201809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:20.214375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:20.219039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:20.293574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82f4da-0527-4964-a940-0d40f7bb3af5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:20.331930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890f4cb-eceb-47ee-ba71-cc15ab3991c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:20.332914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890f4cb-eceb-47ee-ba71-cc15ab3991c4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.333197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890f4cb-eceb-47ee-ba71-cc15ab3991c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.333516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890f4cb-eceb-47ee-ba71-cc15ab3991c4 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.334676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890f4cb-eceb-47ee-ba71-cc15ab3991c4 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:20.337166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890f4cb-eceb-47ee-ba71-cc15ab3991c4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:20.339343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 95/197] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:20 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:20.340859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1ac9-80cc-403a-8dbf-24931203072c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.341142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1ac9-80cc-403a-8dbf-24931203072c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.341457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1ac9-80cc-403a-8dbf-24931203072c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.363118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-c32f1ac9-80cc-403a-8dbf-24931203072c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:20.364475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c32f1ac9-80cc-403a-8dbf-24931203072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:20.422312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:20.443553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 103/198] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:20.444424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:20.452179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 96/199] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:20.453153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f1ac9-80cc-403a-8dbf-24931203072c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:20.481658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:20.562690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48666b6-c36f-4194-ae5f-f3ff06f59c52 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:20.565350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f1ac9-80cc-403a-8dbf-24931203072c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:20.608416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:20.609036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.609334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.609566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:20.609975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.610202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.610313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=c8c7c98cffd340a78515170c150fc447) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:20.610471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.610622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:20.610770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.611861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:20.612071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:20.613325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:20.615154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 97/200] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:20 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:20.618167 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 104/201] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:20 2022] GET /identity/v3/role_assignments?user.id=c8c7c98cffd340a78515170c150fc447&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:20.621889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:20.624109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:20.689723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:20.755245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:20.769238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cdb00-aa75-4344-99f1-f1c899fef22e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:20.803334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efaaf8-79db-4e6b-a1eb-9e55fe2fd43f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:20.807348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:20.808191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.808618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.809033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.809960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] RBAC: Adding query filter params (role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=c8c7c98cffd340a78515170c150fc447) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:20.810450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:20.812059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:20.818168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 105/202] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:20 2022] GET /identity/v3/role_assignments?user.id=c8c7c98cffd340a78515170c150fc447&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=807306af822540d7986dfb93d0b3f344 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:20.876930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:20.877645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:20.877901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:20.878164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:20.895853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:20.901578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:20.902535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:20.903348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 98/203] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:20.908932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:21.054467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac4b5c-e2f8-4817-8106-62ab29027871 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:21.097038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:21.097742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:21.098022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:21.098292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:21.098976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:21.099275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:21.100686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:21.113485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 106/204] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:21.119388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:21.266048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875877c1-317d-46ce-8120-b3e370d2397c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:21.283082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:21.318023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:21.318918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:21.319391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:21.319845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] PATH_INFO: `/v3/users/4eedef18683d42f28aee1079cd0d8aaa` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:21.321214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] RBAC: Authorizing `identity:get_user(user_id=4eedef18683d42f28aee1079cd0d8aaa)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:21.322879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:21.324970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 99/205] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:21 2022] GET /identity/v3/users/4eedef18683d42f28aee1079cd0d8aaa => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:21.330132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:21.356234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:21.464934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ba5a1-b470-42df-83b2-844ebfd9cf67 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:21.503841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:21.504464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:21.504739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:21.505010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:21.602911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:21.604740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:21.626142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:21.729594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 107/206] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:21.735137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:21.865618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9f74d-c1c1-43dc-8295-349c15e43997 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:21.912214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a554e7-c61a-449e-bf5e-d594de1eafaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:21.912843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a554e7-c61a-449e-bf5e-d594de1eafaf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:21.913123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a554e7-c61a-449e-bf5e-d594de1eafaf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:21.913428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a554e7-c61a-449e-bf5e-d594de1eafaf None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:21.917960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a554e7-c61a-449e-bf5e-d594de1eafaf None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:21.919417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a554e7-c61a-449e-bf5e-d594de1eafaf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:21.921067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 100/207] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:21 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:21.926995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:22.046037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc13bb4-9c5c-4cf9-b661-1fbc22da03f5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:22.061284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:22.089307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe08c18-62d4-4c00-bbc7-6324c3948d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:22.089922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08c18-62d4-4c00-bbc7-6324c3948d0f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.090199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08c18-62d4-4c00-bbc7-6324c3948d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.090467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08c18-62d4-4c00-bbc7-6324c3948d0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.091199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe08c18-62d4-4c00-bbc7-6324c3948d0f None admin] RBAC: Adding query filter params (role.id=97e9dc3569a94caeb432886a8e5e708c, scope.project.id=6d75a1525d11453faac47809e5dc2f01, user.id=4eedef18683d42f28aee1079cd0d8aaa) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:22.091512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe08c18-62d4-4c00-bbc7-6324c3948d0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:22.092968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe08c18-62d4-4c00-bbc7-6324c3948d0f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:22.099021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 108/208] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:21 2022] GET /identity/v3/role_assignments?user.id=4eedef18683d42f28aee1079cd0d8aaa&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=97e9dc3569a94caeb432886a8e5e708c => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:22.162728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe990f4-67d0-4a85-95a5-5c4ca33236f1 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.163026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe990f4-67d0-4a85-95a5-5c4ca33236f1 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.163275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe990f4-67d0-4a85-95a5-5c4ca33236f1 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.164715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 101/209] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:22.170310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878821-2252-492a-bc89-6a251e635dc5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.170595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878821-2252-492a-bc89-6a251e635dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.170872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b878821-2252-492a-bc89-6a251e635dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.198454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8b878821-2252-492a-bc89-6a251e635dc5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:22.199697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8b878821-2252-492a-bc89-6a251e635dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:22.274453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f354a-1515-421c-a06a-17834055c958 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.275104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f354a-1515-421c-a06a-17834055c958 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.275594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f354a-1515-421c-a06a-17834055c958 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.277436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 102/210] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:22.283458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f34598-47f4-449e-8e1a-b0f907aa3990 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.284148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f34598-47f4-449e-8e1a-b0f907aa3990 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.284616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f34598-47f4-449e-8e1a-b0f907aa3990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.285293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 109/211] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:22.322376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-04f34598-47f4-449e-8e1a-b0f907aa3990 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:22.323941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-04f34598-47f4-449e-8e1a-b0f907aa3990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:22.333456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b878821-2252-492a-bc89-6a251e635dc5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:22.386610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:22.419154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 103/212] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:22.448467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f34598-47f4-449e-8e1a-b0f907aa3990 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:22.457357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b878821-2252-492a-bc89-6a251e635dc5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:22.473620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:22.499923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:22.500925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.501427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.501922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.513199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:22.518824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:22.519740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:22.520524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:22.524449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 110/213] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:22.534357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:22.571349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f34598-47f4-449e-8e1a-b0f907aa3990 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:22.620059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:22.620785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.621061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.621356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.622566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:22.624147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:22.626053 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 104/214] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:22.631425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:22.647838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dfba8-759e-4682-a24f-966ef0aea66a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:22.678765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:22.686166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:22.687442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.687719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.687977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.688710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:22.689014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:22.690618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:22.699772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 111/215] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:22.705498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:22.736886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90b64e-df32-422e-ab1b-94fc411d475a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:22.738203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:22.772355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:22.772977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.773261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.773645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.774778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:22.776228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:22.804756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 105/216] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:22 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:22.822491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc17c93-8321-4fb5-b05d-5e3074fee33a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:22.860824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683dd55-bfa7-4981-9c9b-c4c1b8e0c0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:22.861520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683dd55-bfa7-4981-9c9b-c4c1b8e0c0fc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:22.861798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683dd55-bfa7-4981-9c9b-c4c1b8e0c0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:22.862066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683dd55-bfa7-4981-9c9b-c4c1b8e0c0fc None admin] PATH_INFO: `/v3/users/ca3fecaa03334199b049e5ac41b52ccd` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:22.877907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683dd55-bfa7-4981-9c9b-c4c1b8e0c0fc None admin] RBAC: Authorizing `identity:get_user(user_id=ca3fecaa03334199b049e5ac41b52ccd)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:22.879413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683dd55-bfa7-4981-9c9b-c4c1b8e0c0fc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:22.881151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 112/217] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:22 2022] GET /identity/v3/users/ca3fecaa03334199b049e5ac41b52ccd => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:22.887622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.001209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:23.006045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a2528-482d-4020-a829-939517ffcd63 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.016997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec4358-7d13-427d-9c6b-488182e3caee None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.017303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec4358-7d13-427d-9c6b-488182e3caee None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.017634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec4358-7d13-427d-9c6b-488182e3caee None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.019149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 113/218] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:23.023933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149b958-3d0e-40f9-a273-f38927825967 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.024285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149b958-3d0e-40f9-a273-f38927825967 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.024627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149b958-3d0e-40f9-a273-f38927825967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.046373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a149b958-3d0e-40f9-a273-f38927825967 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:23.047655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a149b958-3d0e-40f9-a273-f38927825967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:23.049002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:23.049779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.050059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.051139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.131989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 114/219] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:23.133843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:23.135330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:23.153916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149b958-3d0e-40f9-a273-f38927825967 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.177374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:23.252220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:23.256463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 106/220] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:23.261913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.305858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149b958-3d0e-40f9-a273-f38927825967 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.343806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:23.344771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.345229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.345726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.358521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:23.364143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:23.365002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:23.365952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:23.368376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 115/221] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:23.374040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.403518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46686551-6f9e-4706-a172-a0e6fa9bdf1a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.471375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:23.472023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.472397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.472684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.478055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:23.479776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:23.485300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 107/222] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:23 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:23.488182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.500975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:23.531112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d633fc8-ae7c-485f-a9e8-ff07daa46535 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.565350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:23.582701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc54049-9696-49ab-a28b-60c5857713a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:23.583940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc54049-9696-49ab-a28b-60c5857713a3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.584286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc54049-9696-49ab-a28b-60c5857713a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.584624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc54049-9696-49ab-a28b-60c5857713a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.585884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc54049-9696-49ab-a28b-60c5857713a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:23.586891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc54049-9696-49ab-a28b-60c5857713a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:23.589107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc54049-9696-49ab-a28b-60c5857713a3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:23.599376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 116/223] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:23.604655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690ecda-078b-4e3f-96fb-f97d9f882f66 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.604655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690ecda-078b-4e3f-96fb-f97d9f882f66 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.604655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690ecda-078b-4e3f-96fb-f97d9f882f66 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.604655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 117/224] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:23.608741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690ecda-078b-4e3f-96fb-f97d9f882f66 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.621446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96730cc-6f6d-4a4a-9ee8-75382b848d43 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.661179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e1cb2-a720-4f7c-a7d6-11c0d6500f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:23.661826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e1cb2-a720-4f7c-a7d6-11c0d6500f74 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.662111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e1cb2-a720-4f7c-a7d6-11c0d6500f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.662370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e1cb2-a720-4f7c-a7d6-11c0d6500f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.663098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e1cb2-a720-4f7c-a7d6-11c0d6500f74 None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=ca3fecaa03334199b049e5ac41b52ccd) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:23.663408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e1cb2-a720-4f7c-a7d6-11c0d6500f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:23.665320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e1cb2-a720-4f7c-a7d6-11c0d6500f74 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:23.673243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 108/225] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:23 2022] GET /identity/v3/role_assignments?user.id=ca3fecaa03334199b049e5ac41b52ccd&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:23.675449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cac4e1-f0f3-4ced-b60b-eeb2feb1a5bf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.676689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cac4e1-f0f3-4ced-b60b-eeb2feb1a5bf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.678629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cac4e1-f0f3-4ced-b60b-eeb2feb1a5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.698040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-70cac4e1-f0f3-4ced-b60b-eeb2feb1a5bf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:23.699546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-70cac4e1-f0f3-4ced-b60b-eeb2feb1a5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:23.745954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690ecda-078b-4e3f-96fb-f97d9f882f66 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.775413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 109/226] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:23.794954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:23.795581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.795857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.796129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] PATH_INFO: `/v3/users/c8c7c98cffd340a78515170c150fc447` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:23.797383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] RBAC: Authorizing `identity:get_user(user_id=c8c7c98cffd340a78515170c150fc447)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:23.806955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:23.808931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 118/227] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:23 2022] GET /identity/v3/users/c8c7c98cffd340a78515170c150fc447 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:23.818747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cac4e1-f0f3-4ced-b60b-eeb2feb1a5bf None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.819831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:23.844584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:23.905026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:23.950152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae474ab-512e-4ebd-9ad7-b7c3722f4273 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.952940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cac4e1-f0f3-4ced-b60b-eeb2feb1a5bf None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:23.986284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:23.986917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:23.987195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:23.987553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:24.010930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:24.020264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:24.021130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:24.023376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 119/228] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:24.024936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:24.024936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:24.024936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:24.024936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:24.025499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:24.028813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:24.028994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:24.029145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 110/229] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:23 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:24.049923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:24.143337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:24.178968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab04a50-c39f-4cd0-b171-02194a908b71 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:24.219129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:24.219743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:24.220022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:24.220301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:24.220964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:24.221282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:24.222688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:24.243316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 120/230] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:24.248987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:24.262511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08b9d9-5649-42b4-9b30-44c69b8c29a8 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:24.314608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:24.320587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:24.321253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:24.321860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:24.322535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/c8c7c98cffd340a78515170c150fc447/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:24.325104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f691be5ba412783c3ac8c173ed84a, user_id=c8c7c98cffd340a78515170c150fc447, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:24.326836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:24.348186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 111/231] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:24 2022] PUT /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/c8c7c98cffd340a78515170c150fc447/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:24.426936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae7ae-a5f2-4bc5-904e-5df94fb1e4e4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:24.457700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:24.464024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:24.464900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:24.465349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:24.465857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] PATH_INFO: `/v3/users/c8c7c98cffd340a78515170c150fc447` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:24.467306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] RBAC: Authorizing `identity:get_user(user_id=c8c7c98cffd340a78515170c150fc447)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:24.469118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:24.471561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 121/232] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:24 2022] GET /identity/v3/users/c8c7c98cffd340a78515170c150fc447 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:24.478681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:24.608662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644a18f-5db7-4bbc-9740-86eecac0570b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:24.646100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f83f7-4a5c-4112-bd0e-613958f1d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:24.646785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f83f7-4a5c-4112-bd0e-613958f1d155 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:24.647069 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f83f7-4a5c-4112-bd0e-613958f1d155 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:24.647341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f83f7-4a5c-4112-bd0e-613958f1d155 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:24.648466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f83f7-4a5c-4112-bd0e-613958f1d155 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:24.650492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f83f7-4a5c-4112-bd0e-613958f1d155 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:24.652288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 112/233] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:24 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:24.657891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:24.763145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f862ea-1eda-4702-989f-7d7e798ab90f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:24.803500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdb0b4-95b6-4fca-830e-3a25a9a274bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:24.804431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdb0b4-95b6-4fca-830e-3a25a9a274bc None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:24.805398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdb0b4-95b6-4fca-830e-3a25a9a274bc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:24.805814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdb0b4-95b6-4fca-830e-3a25a9a274bc None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/c8c7c98cffd340a78515170c150fc447/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:24.808163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdb0b4-95b6-4fca-830e-3a25a9a274bc None admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f691be5ba412783c3ac8c173ed84a, user_id=c8c7c98cffd340a78515170c150fc447, role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:24.809796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdb0b4-95b6-4fca-830e-3a25a9a274bc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:24.824702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 122/234] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:24 2022] PUT /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/c8c7c98cffd340a78515170c150fc447/roles/807306af822540d7986dfb93d0b3f344 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:18:25.385209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5723a6-e707-43cb-ba51-7c41a9e84567 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:25.386070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5723a6-e707-43cb-ba51-7c41a9e84567 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:25.386492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5723a6-e707-43cb-ba51-7c41a9e84567 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:25.388286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 113/235] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:25.393714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02119b8-88a0-44fb-9b79-c700d04a0670 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:25.394155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02119b8-88a0-44fb-9b79-c700d04a0670 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:25.394588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02119b8-88a0-44fb-9b79-c700d04a0670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:25.413820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d02119b8-88a0-44fb-9b79-c700d04a0670 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:25.415922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d02119b8-88a0-44fb-9b79-c700d04a0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:25.472530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:25.474167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb86d3-6176-4ed7-a09a-e1349c3d7d4d None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:25.474985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb86d3-6176-4ed7-a09a-e1349c3d7d4d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:25.475518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb86d3-6176-4ed7-a09a-e1349c3d7d4d None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:25.478537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 114/236] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:25.486414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d728f-d555-4175-b8ed-53ee3873695b None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:25.487107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d728f-d555-4175-b8ed-53ee3873695b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:25.487957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d728f-d555-4175-b8ed-53ee3873695b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:25.495672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 123/237] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:25.518317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9b1d728f-d555-4175-b8ed-53ee3873695b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:25.519508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9b1d728f-d555-4175-b8ed-53ee3873695b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:25.529078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02119b8-88a0-44fb-9b79-c700d04a0670 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:25.597160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:25.624093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 115/238] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:25.692961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02119b8-88a0-44fb-9b79-c700d04a0670 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:25.700836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d728f-d555-4175-b8ed-53ee3873695b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:25.757372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:25.758989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:25.760235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:25.760709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:25.782457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:25.790828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:25.791719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:25.792517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:25.796154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 124/239] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:25.802406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:25.865685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d728f-d555-4175-b8ed-53ee3873695b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:25.905107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:25.929045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:25.930253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:25.930804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:25.931376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:25.932962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:25.935676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:25.938444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 116/240] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:25.945090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.015973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519804e-5ba7-425d-a7f3-fec5279da14d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.053631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:26.087603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.089652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.090340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.091463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.092535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:26.094368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:26.096925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:26.115866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 125/241] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:26.122235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.127970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57116d-80f5-4132-9ea2-22aa1863c919 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.164797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbad48-99b0-4525-b8c6-f4b36c65a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.165682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbad48-99b0-4525-b8c6-f4b36c65a3fe None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.166104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbad48-99b0-4525-b8c6-f4b36c65a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.166517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbad48-99b0-4525-b8c6-f4b36c65a3fe None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.167315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbad48-99b0-4525-b8c6-f4b36c65a3fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:26.168899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbad48-99b0-4525-b8c6-f4b36c65a3fe None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:26.197312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 117/242] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:25 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:26.198287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84a968-6846-403f-abb7-4d787bbbe71a None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.198570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84a968-6846-403f-abb7-4d787bbbe71a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.199081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84a968-6846-403f-abb7-4d787bbbe71a None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.200213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 118/243] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:26.210836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429474-cc4b-41da-9c96-7bf10e0a72bb None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.211194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429474-cc4b-41da-9c96-7bf10e0a72bb None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.211433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429474-cc4b-41da-9c96-7bf10e0a72bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.255010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-54429474-cc4b-41da-9c96-7bf10e0a72bb None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:26.259521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-54429474-cc4b-41da-9c96-7bf10e0a72bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:26.292753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380409-0071-4e5e-a372-b6cbbc567450 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.317365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:26.359218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.359855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.360130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.360403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] PATH_INFO: `/v3/users/0668a446ecb14fc593b2a1346b345651` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.376441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] RBAC: Authorizing `identity:get_user(user_id=0668a446ecb14fc593b2a1346b345651)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:26.378074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:26.380093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 126/244] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:26 2022] GET /identity/v3/users/0668a446ecb14fc593b2a1346b345651 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:26.387986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.402995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 119/245] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:26.426285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429474-cc4b-41da-9c96-7bf10e0a72bb None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.478485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:26.524015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ddda1-0b7b-4134-b89e-13f7ff49752f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.556551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429474-cc4b-41da-9c96-7bf10e0a72bb None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.570226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.571185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.571833 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.572322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.606847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.607726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.608267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.608706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.617974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:26.622549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:26.628780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:26.629742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:26.630534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:26.633022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 120/246] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:26.638897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.661233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:26.663036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:26.763036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bd3c4-043f-413f-8da4-9b8e8a7b21e1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.765351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:26.793853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 127/247] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:26.799847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.816972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.819431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.820482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.821756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.822934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:26.823511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:26.825638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:26.838506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 121/248] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:26.846418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.881066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:26.926878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20678f88-1d87-4b62-8e05-d9dae1813746 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.960119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e9bd1-a66a-48fd-8f82-0ac2bbd7bc5c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:26.966193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.967065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.967526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.968026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:26.969447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:26.971255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:26.973219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 128/249] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:26 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:26.980267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:26.997659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690df24d-fabe-44a9-ac0b-6be8919a690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:26.998460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690df24d-fabe-44a9-ac0b-6be8919a690e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:26.998806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690df24d-fabe-44a9-ac0b-6be8919a690e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:26.999190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690df24d-fabe-44a9-ac0b-6be8919a690e None admin] PATH_INFO: `/v3/users/ca3fecaa03334199b049e5ac41b52ccd` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.000488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690df24d-fabe-44a9-ac0b-6be8919a690e None admin] RBAC: Authorizing `identity:get_user(user_id=ca3fecaa03334199b049e5ac41b52ccd)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:27.002050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690df24d-fabe-44a9-ac0b-6be8919a690e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:27.004154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 122/250] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:26 2022] GET /identity/v3/users/ca3fecaa03334199b049e5ac41b52ccd => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:27.007045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be8cf4-642c-41c7-b257-cee736a2cf2a None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.007399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be8cf4-642c-41c7-b257-cee736a2cf2a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.007742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be8cf4-642c-41c7-b257-cee736a2cf2a None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.009371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 123/251] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:18:27.010007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be8cf4-642c-41c7-b257-cee736a2cf2a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:27.049289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:27.093153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df020d-d1b0-4b56-9623-feb9801df5fa None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:27.122999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be8cf4-642c-41c7-b257-cee736a2cf2a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:27.130234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8798ecb-a415-47c6-82c2-fd99de809e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:27.131097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8798ecb-a415-47c6-82c2-fd99de809e81 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.131552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8798ecb-a415-47c6-82c2-fd99de809e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.132053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8798ecb-a415-47c6-82c2-fd99de809e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.132996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8798ecb-a415-47c6-82c2-fd99de809e81 None admin] RBAC: Adding query filter params (role.id=97e9dc3569a94caeb432886a8e5e708c, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=0668a446ecb14fc593b2a1346b345651) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:27.133533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8798ecb-a415-47c6-82c2-fd99de809e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:27.135160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8798ecb-a415-47c6-82c2-fd99de809e81 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:27.138490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:27.142419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 129/252] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:26 2022] GET /identity/v3/role_assignments?user.id=0668a446ecb14fc593b2a1346b345651&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=97e9dc3569a94caeb432886a8e5e708c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:27.143809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a4b9a-9a1a-498d-8d03-5b75aa96a148 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.149315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a4b9a-9a1a-498d-8d03-5b75aa96a148 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.150263 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a4b9a-9a1a-498d-8d03-5b75aa96a148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.161556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340847c-c539-4308-9c38-05464379c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:27.162307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340847c-c539-4308-9c38-05464379c45f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.163332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340847c-c539-4308-9c38-05464379c45f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.163332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340847c-c539-4308-9c38-05464379c45f None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.164809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340847c-c539-4308-9c38-05464379c45f None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:27.166382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340847c-c539-4308-9c38-05464379c45f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:27.168204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 124/253] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:27 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:27.178570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340847c-c539-4308-9c38-05464379c45f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:27.190508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-788a4b9a-9a1a-498d-8d03-5b75aa96a148 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:27.192089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-788a4b9a-9a1a-498d-8d03-5b75aa96a148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:27.339008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 130/254] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:27.369753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:27.374466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340847c-c539-4308-9c38-05464379c45f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:27.385168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a4b9a-9a1a-498d-8d03-5b75aa96a148 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:27.454004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a9f2-7eac-41d6-8202-ce10ac574f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:27.454689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a9f2-7eac-41d6-8202-ce10ac574f4e None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.454977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a9f2-7eac-41d6-8202-ce10ac574f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.455262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a9f2-7eac-41d6-8202-ce10ac574f4e None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/ca3fecaa03334199b049e5ac41b52ccd/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.457293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a9f2-7eac-41d6-8202-ce10ac574f4e None admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f691be5ba412783c3ac8c173ed84a, user_id=ca3fecaa03334199b049e5ac41b52ccd, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:27.458756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a9f2-7eac-41d6-8202-ce10ac574f4e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:27.510770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 125/255] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:27 2022] PUT /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/ca3fecaa03334199b049e5ac41b52ccd/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:27.527644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60ff96-023c-4242-98ea-15ed3c9cf94f None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.527954 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60ff96-023c-4242-98ea-15ed3c9cf94f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.528191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60ff96-023c-4242-98ea-15ed3c9cf94f None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.529805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 126/256] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:27.536820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb097-4973-4a1e-8cd1-ac0f5f408990 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.537981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb097-4973-4a1e-8cd1-ac0f5f408990 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.538265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb097-4973-4a1e-8cd1-ac0f5f408990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.597378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-53dbb097-4973-4a1e-8cd1-ac0f5f408990 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:27.597378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-53dbb097-4973-4a1e-8cd1-ac0f5f408990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:27.657310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a4b9a-9a1a-498d-8d03-5b75aa96a148 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:27.725082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 127/257] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:27.756097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbb097-4973-4a1e-8cd1-ac0f5f408990 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:27.781660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:27.810613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:27.811365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.811551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.813300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.845901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:27.850768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:27.851707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:27.853013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:27.860529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 131/258] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:27.862644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:27.883422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:27.893507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbb097-4973-4a1e-8cd1-ac0f5f408990 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:27.937786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:27.939573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:27.939853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:27.940124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:27.953066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:27.968262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:27.970365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:27.972299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:27.977678 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 128/259] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:27.982693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:28.108753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ef888-1e62-4793-b8f3-5f6bc5c7e2ce None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:28.175713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21558e2-bf1a-4aa0-8482-306ad510200b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:28.179051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:28.179689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:28.179969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:28.180233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:28.180958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:28.181259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:28.182704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:28.200893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 132/260] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:28.206756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:28.251141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:28.251789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:28.252068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:28.252336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:28.252995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:28.253310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:28.255108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:28.268911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 129/261] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:28.277906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:28.303529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:28.318780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:28.387737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed66b-88de-4fbb-abc9-7070d3e02820 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:28.400299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd9f6-8012-429a-b986-d6fddc027699 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:28.421748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:28.422380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:28.422659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:28.422924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] PATH_INFO: `/v3/users/c8c7c98cffd340a78515170c150fc447` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:28.424155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] RBAC: Authorizing `identity:get_user(user_id=c8c7c98cffd340a78515170c150fc447)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:28.425872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:28.427791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 130/262] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:28 2022] GET /identity/v3/users/c8c7c98cffd340a78515170c150fc447 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:28.434107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:28.463761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:28.464734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:28.465212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:28.465798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] PATH_INFO: `/v3/users/c8c7c98cffd340a78515170c150fc447` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:28.467245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] RBAC: Authorizing `identity:get_user(user_id=c8c7c98cffd340a78515170c150fc447)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:28.469029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:28.476973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 133/263] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:28 2022] GET /identity/v3/users/c8c7c98cffd340a78515170c150fc447 => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:28.481623 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:28.531781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:28.555953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158f2c5-fa9a-4afe-b7ed-16e7754b2910 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:28.599063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:28.599677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:28.599956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:28.600225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:28.678879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:28.681228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:28.691260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7d5c0-848e-4f1d-95fc-efc3d775a725 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:28.736124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:28.767353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:28.767974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:28.768254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:28.768523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:28.771336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:28.791814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 131/264] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 12:18:28.797448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:28.868323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:28.869841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:28.970607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 134/265] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 12:18:28.973098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd1e71-a4cd-43ca-a67e-4a55105461ce None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:28.978739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:29.021622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:29.045506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:29.048439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.048924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.049470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.052288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:29.055808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:29.061535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 132/266] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:28 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:29.074259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:29.200451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:29.222764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c955-f02e-4384-ba61-7e8e91a01b0b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:29.292784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11401797-af4b-4117-a228-cbe67bc4f3ec None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:29.303065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:29.303705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.304030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.304305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.305491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:29.306954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:29.308666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 135/267] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:28 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:29.317318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:29.365524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14868f19-91d2-4a49-adb1-4375708fa9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:29.366380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14868f19-91d2-4a49-adb1-4375708fa9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.366794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14868f19-91d2-4a49-adb1-4375708fa9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.367236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14868f19-91d2-4a49-adb1-4375708fa9a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.368118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14868f19-91d2-4a49-adb1-4375708fa9a6 None admin] RBAC: Adding query filter params (role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=c8c7c98cffd340a78515170c150fc447) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:29.368565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14868f19-91d2-4a49-adb1-4375708fa9a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:29.371207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14868f19-91d2-4a49-adb1-4375708fa9a6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:29.388547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 133/268] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:29 2022] GET /identity/v3/role_assignments?user.id=c8c7c98cffd340a78515170c150fc447&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=807306af822540d7986dfb93d0b3f344 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:29.463475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:29.480904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd2e26-268e-429b-ba07-63e495955113 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:29.520120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6ed65-ce39-4c7b-aced-9e576191f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:29.521004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ed65-ce39-4c7b-aced-9e576191f3ee None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.521458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ed65-ce39-4c7b-aced-9e576191f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.521868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ed65-ce39-4c7b-aced-9e576191f3ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.522792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6ed65-ce39-4c7b-aced-9e576191f3ee None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=c8c7c98cffd340a78515170c150fc447) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:29.523242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6ed65-ce39-4c7b-aced-9e576191f3ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:29.525499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6ed65-ce39-4c7b-aced-9e576191f3ee None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:29.533107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 136/269] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:29 2022] GET /identity/v3/role_assignments?user.id=c8c7c98cffd340a78515170c150fc447&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:29.804419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b08f8-b80e-45ff-a3e2-baf4fb202554 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.806014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b08f8-b80e-45ff-a3e2-baf4fb202554 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.806488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b08f8-b80e-45ff-a3e2-baf4fb202554 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.808083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 134/270] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:29.813382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525603-cc5f-472e-a7f6-ba0cbabb5609 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.813888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525603-cc5f-472e-a7f6-ba0cbabb5609 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.814373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525603-cc5f-472e-a7f6-ba0cbabb5609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.835399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-60525603-cc5f-472e-a7f6-ba0cbabb5609 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:29.836946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-60525603-cc5f-472e-a7f6-ba0cbabb5609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:29.837052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81c9b-5b33-4c3c-b851-f7f17abc7251 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.837329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81c9b-5b33-4c3c-b851-f7f17abc7251 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.837639 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81c9b-5b33-4c3c-b851-f7f17abc7251 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.839277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 135/271] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:29.844037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23eb3-425a-4bbc-9d77-56a75c9b2573 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:29.844321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23eb3-425a-4bbc-9d77-56a75c9b2573 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:29.844600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b23eb3-425a-4bbc-9d77-56a75c9b2573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:29.866567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-67b23eb3-425a-4bbc-9d77-56a75c9b2573 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:29.868326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-67b23eb3-425a-4bbc-9d77-56a75c9b2573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:29.924540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 137/272] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:29.948153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 136/273] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:29.958537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60525603-cc5f-472e-a7f6-ba0cbabb5609 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:29.973571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b23eb3-425a-4bbc-9d77-56a75c9b2573 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.039581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.060895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.084655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60525603-cc5f-472e-a7f6-ba0cbabb5609 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.102461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b23eb3-425a-4bbc-9d77-56a75c9b2573 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.126906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68406c18-ff1b-4a88-81f4-5666798249b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.127535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68406c18-ff1b-4a88-81f4-5666798249b9 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.127813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68406c18-ff1b-4a88-81f4-5666798249b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.128081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68406c18-ff1b-4a88-81f4-5666798249b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.129168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68406c18-ff1b-4a88-81f4-5666798249b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:30.130735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68406c18-ff1b-4a88-81f4-5666798249b9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:30.132340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 138/274] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:30.133507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21233582-3d14-4295-9f48-488d525b51f5 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.133783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21233582-3d14-4295-9f48-488d525b51f5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.134052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21233582-3d14-4295-9f48-488d525b51f5 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.135373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 139/275] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:30.139602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21233582-3d14-4295-9f48-488d525b51f5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.143899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06737d5f-bbc1-4e95-821c-ca03f52d094c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.144531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737d5f-bbc1-4e95-821c-ca03f52d094c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.144805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737d5f-bbc1-4e95-821c-ca03f52d094c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.145071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06737d5f-bbc1-4e95-821c-ca03f52d094c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.155959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06737d5f-bbc1-4e95-821c-ca03f52d094c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-06737d5f-bbc1-4e95-821c-ca03f52d094c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:30.160631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:30.161550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:30.162359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 137/276] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:30.164680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47bdb5-26cf-4ed3-8f65-a976ed8fc1cf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.164680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47bdb5-26cf-4ed3-8f65-a976ed8fc1cf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.164680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47bdb5-26cf-4ed3-8f65-a976ed8fc1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.181514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3d47bdb5-26cf-4ed3-8f65-a976ed8fc1cf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:30.182677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3d47bdb5-26cf-4ed3-8f65-a976ed8fc1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:30.248113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21233582-3d14-4295-9f48-488d525b51f5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.258275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 138/277] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:30.258275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47bdb5-26cf-4ed3-8f65-a976ed8fc1cf None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.289583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.290486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.290931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.291362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.292199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:30.293844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:30.297178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.316006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 140/278] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:30 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:18:30.316722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.326515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.401110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47bdb5-26cf-4ed3-8f65-a976ed8fc1cf None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.446268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f3922-a4e6-4dcc-b442-533795c55a10 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.448774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.449679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.450135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.450556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.451384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:30.451837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:30.453448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:30.464309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 139/279] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:30.470359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.490495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.491545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.491911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.492278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.505880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:30.512678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:30.513649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:30.514643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 141/280] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:30.520081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.578171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.585839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d96e-a93f-4121-8a10-1f1aefda7ba4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.606745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.641800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.642734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.643020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.643291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] PATH_INFO: `/v3/users/0668a446ecb14fc593b2a1346b345651` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.645744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] RBAC: Authorizing `identity:get_user(user_id=0668a446ecb14fc593b2a1346b345651)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:30.647070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:30.647271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ef7a9-7a14-4191-aaf3-fdc932475644 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.649169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 140/281] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:30 2022] GET /identity/v3/users/0668a446ecb14fc593b2a1346b345651 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:30.657458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.688311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.689087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.689586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.689958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.690834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:30.691232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:30.692703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:30.703106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 142/282] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:30.710897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.789782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc552-3f3f-413b-88e6-1143be7e5a33 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.826569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fee34-efe0-488e-8d91-e28c13956ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.827179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fee34-efe0-488e-8d91-e28c13956ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.827537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fee34-efe0-488e-8d91-e28c13956ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.827884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fee34-efe0-488e-8d91-e28c13956ce2 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.829080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fee34-efe0-488e-8d91-e28c13956ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:30.830623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fee34-efe0-488e-8d91-e28c13956ce2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:30.832974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 141/283] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:30 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:30.834334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286602f-a59f-475d-8ce5-375c44467130 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.834702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286602f-a59f-475d-8ce5-375c44467130 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.834977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286602f-a59f-475d-8ce5-375c44467130 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.836236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 142/284] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:30.838953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286602f-a59f-475d-8ce5-375c44467130 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:30.848272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836a1e-532f-439c-b0b1-50da2bb33a78 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:30.877563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.877869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:30.888064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2c0c4-5970-42cc-9dd4-e94f31bef1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:30.888702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2c0c4-5970-42cc-9dd4-e94f31bef1df None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.889033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2c0c4-5970-42cc-9dd4-e94f31bef1df None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.889382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2c0c4-5970-42cc-9dd4-e94f31bef1df None admin] PATH_INFO: `/v3/users/0ea7a155edf04385a3ca82bf01e97727` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.903940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2c0c4-5970-42cc-9dd4-e94f31bef1df None admin] RBAC: Authorizing `identity:get_user(user_id=0ea7a155edf04385a3ca82bf01e97727)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:30.905534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2c0c4-5970-42cc-9dd4-e94f31bef1df None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:30.907628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 143/285] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:30 2022] GET /identity/v3/users/0ea7a155edf04385a3ca82bf01e97727 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:30.908959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e565e-be44-412e-8a7e-2ebc2e3f3e26 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:30.909244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e565e-be44-412e-8a7e-2ebc2e3f3e26 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:30.909540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e565e-be44-412e-8a7e-2ebc2e3f3e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:30.927533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d26e565e-be44-412e-8a7e-2ebc2e3f3e26 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:30.928717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d26e565e-be44-412e-8a7e-2ebc2e3f3e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:30.969795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286602f-a59f-475d-8ce5-375c44467130 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:31.009378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 144/286] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:31.011318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e565e-be44-412e-8a7e-2ebc2e3f3e26 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:31.020083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:31.021047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:31.021563 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:31.022057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/0668a446ecb14fc593b2a1346b345651/roles/97e9dc3569a94caeb432886a8e5e708c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:31.024332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f691be5ba412783c3ac8c173ed84a, user_id=0668a446ecb14fc593b2a1346b345651, role_id=97e9dc3569a94caeb432886a8e5e708c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:31.026068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:31.049893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 143/287] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:18:30 2022] PUT /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/users/0668a446ecb14fc593b2a1346b345651/roles/97e9dc3569a94caeb432886a8e5e708c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:18:31.054177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:31.202704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e565e-be44-412e-8a7e-2ebc2e3f3e26 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:31.221439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:31.240685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:31.244804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:31.245462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:31.245772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:31.246044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:31.246852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, user.id=0ea7a155edf04385a3ca82bf01e97727, scope.system=all) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:31.247240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:31.248890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:31.254604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 145/288] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 10 12:18:31 2022] GET /identity/v3/role_assignments?user.id=0ea7a155edf04385a3ca82bf01e97727&scope.system=all&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:31.265761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b37fb7-dd74-438b-978a-1fda9b5ca441 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:31.318392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:31.319059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:31.319341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:31.319616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:31.337971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:31.343736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:31.344696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:31.345517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 144/289] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:31.351325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:31.496277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9570b-2d69-43e7-ae11-28986097e7d6 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:31.536842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:31.537820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:31.538280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:31.538748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:31.540156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:31.540638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:31.542256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:31.551001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 146/290] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:31.556866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:31.700022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cdb83-5f0e-4d26-b7a3-ad912bcd2144 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:31.746618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:31.747175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:31.747455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:31.747728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] PATH_INFO: `/v3/users/ca3fecaa03334199b049e5ac41b52ccd` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:31.748890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] RBAC: Authorizing `identity:get_user(user_id=ca3fecaa03334199b049e5ac41b52ccd)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:31.750376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:31.752274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 145/291] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:31 2022] GET /identity/v3/users/ca3fecaa03334199b049e5ac41b52ccd => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:31.758282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:31.831771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:31.927334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3710c-d865-47bf-8f57-8e042b550512 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:31.965151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:31.966565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:31.967068 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:31.967537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:32.038075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:32.039912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:32.083059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:32.180860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 147/292] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:32.190506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:32.221424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:32.330389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6f56b-1836-44c5-9603-63213dff3276 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:32.363993 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067483b2-ec75-4248-a972-d75b07a13845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:32.364629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067483b2-ec75-4248-a972-d75b07a13845 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:32.364903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067483b2-ec75-4248-a972-d75b07a13845 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:32.365173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067483b2-ec75-4248-a972-d75b07a13845 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:32.366324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067483b2-ec75-4248-a972-d75b07a13845 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:32.368303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067483b2-ec75-4248-a972-d75b07a13845 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:32.370265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 146/293] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:32 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:32.376112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:32.553993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebba04-1284-4b4b-b559-608c72903178 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:32.607839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969eb117-9970-47c7-aa53-a9abf8e425b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:32.608465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eb117-9970-47c7-aa53-a9abf8e425b0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:32.608742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eb117-9970-47c7-aa53-a9abf8e425b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:32.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eb117-9970-47c7-aa53-a9abf8e425b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:32.609813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969eb117-9970-47c7-aa53-a9abf8e425b0 None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=ca3fecaa03334199b049e5ac41b52ccd) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:32.610096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969eb117-9970-47c7-aa53-a9abf8e425b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:32.611545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969eb117-9970-47c7-aa53-a9abf8e425b0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:32.618557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 148/294] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:32 2022] GET /identity/v3/role_assignments?user.id=ca3fecaa03334199b049e5ac41b52ccd&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:32.879893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d6f3-8445-4271-b0f1-99b68d0f3347 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:32.880542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d6f3-8445-4271-b0f1-99b68d0f3347 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:32.881019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d6f3-8445-4271-b0f1-99b68d0f3347 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:32.882592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 147/295] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:32.888616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9a02-279f-4dde-ac9f-7baab798690c None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:32.888900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9a02-279f-4dde-ac9f-7baab798690c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:32.889179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9a02-279f-4dde-ac9f-7baab798690c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:32.904585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:32.913509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-166c9a02-279f-4dde-ac9f-7baab798690c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:32.915420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-166c9a02-279f-4dde-ac9f-7baab798690c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:32.987089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 149/296] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:33.011858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d6f3-8445-4271-b0f1-99b68d0f3347 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.052670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9302-cbc5-4bcc-af82-8b36c4494528 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.053300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9302-cbc5-4bcc-af82-8b36c4494528 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.053776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9302-cbc5-4bcc-af82-8b36c4494528 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.056087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 150/297] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:33.061749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7e59-bacd-46eb-aad4-09e6c39ed0e7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.062269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7e59-bacd-46eb-aad4-09e6c39ed0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.062741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7e59-bacd-46eb-aad4-09e6c39ed0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.085947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e09d7e59-bacd-46eb-aad4-09e6c39ed0e7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:33.087576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e09d7e59-bacd-46eb-aad4-09e6c39ed0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:33.119486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:33.141474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d6f3-8445-4271-b0f1-99b68d0f3347 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:33.169385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 151/298] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:33.178584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:33.179433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.180438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.180848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.191824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d7e59-bacd-46eb-aad4-09e6c39ed0e7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.198462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:33.205179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:33.206121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:33.206922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 148/299] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:33.212147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.271078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:33.314528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d7e59-bacd-46eb-aad4-09e6c39ed0e7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:33.329422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3a4d-676a-4d74-9f34-01a2c09e8130 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:33.359798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:33.360727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.361010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.361724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.363099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:33.365172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:33.368108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 152/300] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:33.374085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.375041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:33.375662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.376224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.376505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.377175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:33.377510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:33.379105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:33.388575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 149/301] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:33.393655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.404725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:33.489130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037b07-9a23-4f97-b41b-71650a746b0a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:33.501850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c848361-aa10-4cac-821b-276d29a44e04 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:33.524981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45636a-339f-42e7-a843-e8db47980ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:33.525618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45636a-339f-42e7-a843-e8db47980ada None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.525895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45636a-339f-42e7-a843-e8db47980ada None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.526161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45636a-339f-42e7-a843-e8db47980ada None admin] PATH_INFO: `/v3/groups` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.526836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45636a-339f-42e7-a843-e8db47980ada None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:33.528274 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45636a-339f-42e7-a843-e8db47980ada None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:33.532481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:33.540598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:33.541314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.541611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.541879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] PATH_INFO: `/v3/users/005dedb64fb8476b849ca5b24feaf920` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.546206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 153/302] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 10 12:18:33 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:18:33.551370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290f21-b328-4232-a6e1-ee9f3ca709a9 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.552236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290f21-b328-4232-a6e1-ee9f3ca709a9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.552794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290f21-b328-4232-a6e1-ee9f3ca709a9 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.555631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 154/303] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:33.558650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] RBAC: Authorizing `identity:get_user(user_id=005dedb64fb8476b849ca5b24feaf920)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:33.560720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:33.563083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428474fb-a0e6-499b-a603-fc8bd9dfd297 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.563638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428474fb-a0e6-499b-a603-fc8bd9dfd297 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.564137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428474fb-a0e6-499b-a603-fc8bd9dfd297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.567198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 150/304] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/users/005dedb64fb8476b849ca5b24feaf920 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:33.574107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.588401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-428474fb-a0e6-499b-a603-fc8bd9dfd297 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:33.590106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-428474fb-a0e6-499b-a603-fc8bd9dfd297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:33.670109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:33.672887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 155/305] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:33.698227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428474fb-a0e6-499b-a603-fc8bd9dfd297 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.710910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c412a7-1260-4d2c-9d85-f942cbd88290 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:33.751827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e761655-49df-456a-9e25-d74601ac8eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:33.752170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761655-49df-456a-9e25-d74601ac8eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.752457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761655-49df-456a-9e25-d74601ac8eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.752734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761655-49df-456a-9e25-d74601ac8eb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.753511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e761655-49df-456a-9e25-d74601ac8eb3 None admin] RBAC: Adding query filter params (role.id=97e9dc3569a94caeb432886a8e5e708c, user.id=005dedb64fb8476b849ca5b24feaf920, scope.system=all) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:33.753829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e761655-49df-456a-9e25-d74601ac8eb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:33.755284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e761655-49df-456a-9e25-d74601ac8eb3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:33.761358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 151/306] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/role_assignments?user.id=005dedb64fb8476b849ca5b24feaf920&scope.system=all&role.id=97e9dc3569a94caeb432886a8e5e708c => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:33.767227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bcb01-1311-4278-97fe-52051b957ec7 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.767512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bcb01-1311-4278-97fe-52051b957ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.767789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bcb01-1311-4278-97fe-52051b957ec7 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.769166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 152/307] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:33.778828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7320ba-a115-4a0c-8260-b09123fa96d8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.779150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7320ba-a115-4a0c-8260-b09123fa96d8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.779352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7320ba-a115-4a0c-8260-b09123fa96d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.798624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ad7320ba-a115-4a0c-8260-b09123fa96d8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:33.799850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ad7320ba-a115-4a0c-8260-b09123fa96d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:33.827285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428474fb-a0e6-499b-a603-fc8bd9dfd297 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:33.854494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:33.866460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:33.868007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:33.868517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:33.868986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:33.882063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 153/308] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:33.885188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:33.890228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:33.891130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:33.891910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:33.899300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 156/309] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:33.901007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7320ba-a115-4a0c-8260-b09123fa96d8 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.904439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:33.991348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:34.021407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0ac1d-16a9-44e7-b3dd-e0eb3e26ec1a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.044955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7320ba-a115-4a0c-8260-b09123fa96d8 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.066536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.068031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.068768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.069299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.081805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.082419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.082699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.082962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.083768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:34.084100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.084789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:34.085660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:34.091911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:34.092796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:34.093628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:34.100157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 154/311] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:34.100598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 157/311] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:34.102403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:34.102852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:34.153594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:34.216609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852dd2-3853-4dea-b0cc-c8db52f1e0d0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.226872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40aa04-7466-4997-82a9-7d8e77bac19e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.256278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:34.263968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.264608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.264887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.265157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] PATH_INFO: `/v3/users/0668a446ecb14fc593b2a1346b345651` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.268795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=0668a446ecb14fc593b2a1346b345651)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.272644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.276214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.277078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.277595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.278114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.278943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 158/312] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:34 2022] GET /identity/v3/users/0668a446ecb14fc593b2a1346b345651 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:34.279220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] RBAC: Adding query filter params (name=member) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:34.279733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.280283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:34.281654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.293700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 155/313] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:34.299893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:34.395492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70739e-3c51-4b27-bb30-64c7fb808fa9 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.409928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:34.434024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e6cb7-c9ff-4af8-b457-3b8fb2c9e752 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.434977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.435657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.436008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.436368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.470458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.471388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.471844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.472317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] PATH_INFO: `/v3/users/0ea7a155edf04385a3ca82bf01e97727` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.473647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] RBAC: Authorizing `identity:get_user(user_id=0ea7a155edf04385a3ca82bf01e97727)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.475276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.477651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 156/314] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:34 2022] GET /identity/v3/users/0ea7a155edf04385a3ca82bf01e97727 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:34.484237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:34.519628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.521998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.555515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:34.610303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3475b2f-b490-4a12-8921-fbd2a75f29b9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.630507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 159/315] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:34.631873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:34.645412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:34.649020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.649930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.650410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.650881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] PATH_INFO: `/v3/system/users/0ea7a155edf04385a3ca82bf01e97727/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.652628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0ea7a155edf04385a3ca82bf01e97727, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.655089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.667083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 157/316] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 10 12:18:34 2022] PUT /identity/v3/system/users/0ea7a155edf04385a3ca82bf01e97727/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:18:34.750001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e832ad-ed9a-4e3d-84ee-675cb876db13 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.789821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce250b8-c528-492e-917e-88adcc1c5451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.790733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce250b8-c528-492e-917e-88adcc1c5451 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.791009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce250b8-c528-492e-917e-88adcc1c5451 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.791278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce250b8-c528-492e-917e-88adcc1c5451 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.792573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce250b8-c528-492e-917e-88adcc1c5451 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.794124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce250b8-c528-492e-917e-88adcc1c5451 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.795872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 160/317] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:34 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:34.802103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:34.924208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a81b06-b9aa-4c1d-b663-4574a4490e76 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:34.941523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:34.963232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a311a-f0e6-4645-bafa-1306af650e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:34.963901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a311a-f0e6-4645-bafa-1306af650e40 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:34.964186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a311a-f0e6-4645-bafa-1306af650e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:34.964459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a311a-f0e6-4645-bafa-1306af650e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:34.965184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a311a-f0e6-4645-bafa-1306af650e40 None admin] RBAC: Adding query filter params (role.id=97e9dc3569a94caeb432886a8e5e708c, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a, user.id=0668a446ecb14fc593b2a1346b345651) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:34.965521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a311a-f0e6-4645-bafa-1306af650e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:34.967010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a311a-f0e6-4645-bafa-1306af650e40 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:34.973309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 158/318] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:18:34 2022] GET /identity/v3/role_assignments?user.id=0668a446ecb14fc593b2a1346b345651&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=97e9dc3569a94caeb432886a8e5e708c => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:35.664608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b14fc0-31e1-4930-b95e-55cd409b1ceb None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:35.665102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b14fc0-31e1-4930-b95e-55cd409b1ceb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:35.665208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b14fc0-31e1-4930-b95e-55cd409b1ceb None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:35.666660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 161/319] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:35.671462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b64d7b-4726-47d1-aa7d-0db7298123f8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:35.671826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b64d7b-4726-47d1-aa7d-0db7298123f8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:35.672016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b64d7b-4726-47d1-aa7d-0db7298123f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:35.690247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-32b64d7b-4726-47d1-aa7d-0db7298123f8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:35.691414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-32b64d7b-4726-47d1-aa7d-0db7298123f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:35.759827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 159/320] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:35.780937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b14fc0-31e1-4930-b95e-55cd409b1ceb None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:35.798213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9071dbd-0778-4fd1-ab3e-be9da7b9ad16 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:35.798511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9071dbd-0778-4fd1-ab3e-be9da7b9ad16 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:35.798820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9071dbd-0778-4fd1-ab3e-be9da7b9ad16 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:35.800399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 160/321] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:18:35.805571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca19408-0c07-4532-8e3e-a740b709ee6a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:35.805852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca19408-0c07-4532-8e3e-a740b709ee6a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:35.806130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca19408-0c07-4532-8e3e-a740b709ee6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:35.825218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-0ca19408-0c07-4532-8e3e-a740b709ee6a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:35.826434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0ca19408-0c07-4532-8e3e-a740b709ee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:35.876791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:35.899204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 161/322] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:35.899866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b14fc0-31e1-4930-b95e-55cd409b1ceb None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:35.921590 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca19408-0c07-4532-8e3e-a740b709ee6a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:35.938593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:35.939277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:35.939554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:35.939826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:35.940915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:35.942388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:35.943936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 162/323] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:35.949713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:35.965040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:36.034044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca19408-0c07-4532-8e3e-a740b709ee6a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:36.057379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8451c8e-83f7-4929-b42b-abe6b99e2a87 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:36.068429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:36.069058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:36.069372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:36.069648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:36.079754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:36.084480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:36.085386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:36.086677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 162/324] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:36.091756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:36.099019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:36.099790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:36.100205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:36.101019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] PATH_INFO: `/v3/groups` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:36.102377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:36.104905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:36.124382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 163/325] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 10 12:18:35 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:18:36.200414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:36.204351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a43ca9-9399-4970-9b14-e761ec385c85 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:36.243041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:36.243662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:36.243944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:36.244219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:36.244902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:36.245207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:36.246648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:36.256022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 163/326] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:36.262093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:36.275202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:36.369587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd589f7-738c-43cd-ac1d-9cc3cbf8a269 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:36.406052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97f8d9-688f-4455-970e-84e1941e4d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:36.406922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f8d9-688f-4455-970e-84e1941e4d21 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:36.407338 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f8d9-688f-4455-970e-84e1941e4d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:36.407747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f8d9-688f-4455-970e-84e1941e4d21 None admin] PATH_INFO: `/v3/users/005dedb64fb8476b849ca5b24feaf920` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:36.409068 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97f8d9-688f-4455-970e-84e1941e4d21 None admin] RBAC: Authorizing `identity:get_user(user_id=005dedb64fb8476b849ca5b24feaf920)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:36.410696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97f8d9-688f-4455-970e-84e1941e4d21 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:36.412405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 164/327] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:36 2022] GET /identity/v3/users/005dedb64fb8476b849ca5b24feaf920 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:36.418229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:36.529694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1c34-d134-4318-848c-544c0791f360 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:36.566716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7da55-95c2-412a-821b-71797f39185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:36.567730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7da55-95c2-412a-821b-71797f39185f None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:36.568206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7da55-95c2-412a-821b-71797f39185f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:36.568703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7da55-95c2-412a-821b-71797f39185f None admin] PATH_INFO: `/v3/system/users/005dedb64fb8476b849ca5b24feaf920/roles/97e9dc3569a94caeb432886a8e5e708c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:36.570870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7da55-95c2-412a-821b-71797f39185f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=005dedb64fb8476b849ca5b24feaf920, role_id=97e9dc3569a94caeb432886a8e5e708c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:36.572460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7da55-95c2-412a-821b-71797f39185f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:36.584707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 164/328] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 10 12:18:36 2022] PUT /identity/v3/system/users/005dedb64fb8476b849ca5b24feaf920/roles/97e9dc3569a94caeb432886a8e5e708c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:36.774492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c108-e6df-4254-865e-a3e08a05f1e4 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:36.775468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c108-e6df-4254-865e-a3e08a05f1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:36.775943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54c108-e6df-4254-865e-a3e08a05f1e4 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:36.777789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 165/329] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:36.783538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7375e-9847-4232-8951-21c9503df9f0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:36.784366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7375e-9847-4232-8951-21c9503df9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:36.784834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7375e-9847-4232-8951-21c9503df9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:36.806227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b0b7375e-9847-4232-8951-21c9503df9f0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:36.807431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b0b7375e-9847-4232-8951-21c9503df9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:36.831102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:36.889367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 165/330] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:36.913625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c108-e6df-4254-865e-a3e08a05f1e4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:37.028815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:37.075060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54c108-e6df-4254-865e-a3e08a05f1e4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:37.121140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:37.121811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:37.122090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:37.122532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:37.136652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:37.141897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:37.142731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:37.143516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 166/331] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:37.151958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7375e-9847-4232-8951-21c9503df9f0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:37.278793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7375e-9847-4232-8951-21c9503df9f0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:37.327826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:37.328369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:37.328656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:37.328934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:37.329639 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:37.329946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:37.331342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:37.340543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 166/332] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:37.350379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:37.494492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af358d-58d4-4684-b466-ba2784710ef2 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:37.539579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c7d87-df5e-4638-b8f6-214c4bb3e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:37.540215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c7d87-df5e-4638-b8f6-214c4bb3e99e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:37.540494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c7d87-df5e-4638-b8f6-214c4bb3e99e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:37.540773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c7d87-df5e-4638-b8f6-214c4bb3e99e None admin] PATH_INFO: `/v3/users/0ea7a155edf04385a3ca82bf01e97727` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:37.541950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c7d87-df5e-4638-b8f6-214c4bb3e99e None admin] RBAC: Authorizing `identity:get_user(user_id=0ea7a155edf04385a3ca82bf01e97727)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:37.543416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c7d87-df5e-4638-b8f6-214c4bb3e99e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:37.545364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 167/333] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:37 2022] GET /identity/v3/users/0ea7a155edf04385a3ca82bf01e97727 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:37.550952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:37.598149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:37.697715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11beb2-8511-4a22-b9cb-9517a7074ed2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:37.742219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaad7f7-1fb3-4ed6-84cf-23eec5f965fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:37.742836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad7f7-1fb3-4ed6-84cf-23eec5f965fa None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:37.743117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad7f7-1fb3-4ed6-84cf-23eec5f965fa None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:37.743389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaad7f7-1fb3-4ed6-84cf-23eec5f965fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:37.744117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaad7f7-1fb3-4ed6-84cf-23eec5f965fa None admin] RBAC: Adding query filter params (role.id=ae3c9f4504f84e25bc89e8c30397aedc, user.id=0ea7a155edf04385a3ca82bf01e97727, scope.system=all) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:37.744429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaad7f7-1fb3-4ed6-84cf-23eec5f965fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:37.745905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaad7f7-1fb3-4ed6-84cf-23eec5f965fa None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:37.768823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 167/334] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 10 12:18:37 2022] GET /identity/v3/role_assignments?user.id=0ea7a155edf04385a3ca82bf01e97727&scope.system=all&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:38.889946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aa2a2-b88b-404b-9737-d38b4155c142 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:38.890598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aa2a2-b88b-404b-9737-d38b4155c142 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:38.890598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aa2a2-b88b-404b-9737-d38b4155c142 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:38.891891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 168/335] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:38.897512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e246-3960-4449-8b0c-6950c07fbeac None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:38.898010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e246-3960-4449-8b0c-6950c07fbeac None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:38.898223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e246-3960-4449-8b0c-6950c07fbeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:38.923046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a038e246-3960-4449-8b0c-6950c07fbeac None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:38.924268 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a038e246-3960-4449-8b0c-6950c07fbeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:39.019015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 168/336] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:39.058805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344aa2a2-b88b-404b-9737-d38b4155c142 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:39.068927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:39.122243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa622ff-ca98-4763-b9c8-78cb44467283 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.122556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa622ff-ca98-4763-b9c8-78cb44467283 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.122807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa622ff-ca98-4763-b9c8-78cb44467283 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.124446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 169/337] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:39.129573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f67e7-320a-4167-b407-31d01e87aed9 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.129864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f67e7-320a-4167-b407-31d01e87aed9 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.130141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f67e7-320a-4167-b407-31d01e87aed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.156101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:39.170307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-726f67e7-320a-4167-b407-31d01e87aed9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:39.171518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-726f67e7-320a-4167-b407-31d01e87aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:39.211390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344aa2a2-b88b-404b-9737-d38b4155c142 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:39.253626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:39.254264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.254538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.254810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.266823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:39.272391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:39.274065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:39.274959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:39.277300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 169/338] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:39.283698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:39.315226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 170/339] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 12:18:39.352368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f67e7-320a-4167-b407-31d01e87aed9 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:39.393468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:39.442903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f9ef7-5e7a-4a6f-8fd7-7ba7af0ff60d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:39.483973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb17ed-9d26-4fba-a9b4-7291732d91bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:39.484582 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb17ed-9d26-4fba-a9b4-7291732d91bb None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.484864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb17ed-9d26-4fba-a9b4-7291732d91bb None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.485132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb17ed-9d26-4fba-a9b4-7291732d91bb None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.485840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb17ed-9d26-4fba-a9b4-7291732d91bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:39.486142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb17ed-9d26-4fba-a9b4-7291732d91bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:39.487548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb17ed-9d26-4fba-a9b4-7291732d91bb None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:39.505781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 170/340] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:39.507124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebb53b-123e-42cc-abee-33b00b433b5b None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.507937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebb53b-123e-42cc-abee-33b00b433b5b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.508225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebb53b-123e-42cc-abee-33b00b433b5b None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.509585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 171/341] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:39.514312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebb53b-123e-42cc-abee-33b00b433b5b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:39.540816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f67e7-320a-4167-b407-31d01e87aed9 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:39.625104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebb53b-123e-42cc-abee-33b00b433b5b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:39.627990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5612e-9d71-483a-96ab-d97c235c4fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:39.628896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5612e-9d71-483a-96ab-d97c235c4fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.629395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5612e-9d71-483a-96ab-d97c235c4fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.629857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5612e-9d71-483a-96ab-d97c235c4fa0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.638428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:39.655518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:39.656350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5612e-9d71-483a-96ab-d97c235c4fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f3c5612e-9d71-483a-96ab-d97c235c4fa0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:39.661601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:39.662546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:39.663322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:39.670158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 171/342] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:39.674732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ca93b-6f37-48da-87ae-17f9d392b929 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.674732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ca93b-6f37-48da-87ae-17f9d392b929 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.674732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ca93b-6f37-48da-87ae-17f9d392b929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.675626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10151f6b-8799-401a-9f69-c298720fb924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:39.676247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10151f6b-8799-401a-9f69-c298720fb924 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.676527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10151f6b-8799-401a-9f69-c298720fb924 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.676800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10151f6b-8799-401a-9f69-c298720fb924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.710946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f04ca93b-6f37-48da-87ae-17f9d392b929 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:39.712109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f04ca93b-6f37-48da-87ae-17f9d392b929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:39.817920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10151f6b-8799-401a-9f69-c298720fb924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:39.819643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10151f6b-8799-401a-9f69-c298720fb924 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:39.837307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 172/343] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:39.838755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ca93b-6f37-48da-87ae-17f9d392b929 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:39.930228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 172/344] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:39.937057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931394-9728-4ec1-91ee-098bc2916d6f None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:39.937384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931394-9728-4ec1-91ee-098bc2916d6f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:39.937653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42931394-9728-4ec1-91ee-098bc2916d6f None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:39.939093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 173/345] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:39.939732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42931394-9728-4ec1-91ee-098bc2916d6f None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:39.957837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ca93b-6f37-48da-87ae-17f9d392b929 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:39.972674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:39.990202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:40.001932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.002831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.003284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.003824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.004762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:40.005238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:40.006923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:40.016733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 173/346] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:40.017866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.052193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42931394-9728-4ec1-91ee-098bc2916d6f None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.089212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a4c75-f1cf-413f-b9b0-a64da7d15234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.090282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a4c75-f1cf-413f-b9b0-a64da7d15234 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.090570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a4c75-f1cf-413f-b9b0-a64da7d15234 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.090847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a4c75-f1cf-413f-b9b0-a64da7d15234 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.105628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a4c75-f1cf-413f-b9b0-a64da7d15234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-423a4c75-f1cf-413f-b9b0-a64da7d15234 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:40.110664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:40.111557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:40.112433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 174/347] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:40.113907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981221-bc5d-4357-bab4-e6e471e04aba None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.113907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981221-bc5d-4357-bab4-e6e471e04aba None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.114077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981221-bc5d-4357-bab4-e6e471e04aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.137850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ee981221-bc5d-4357-bab4-e6e471e04aba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:40.139317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ee981221-bc5d-4357-bab4-e6e471e04aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:40.141033 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d928eb-36ac-4b39-a622-12c7bb861366 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.181841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.183060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.183689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.184231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.185638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:40.188517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:40.190381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 174/348] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:40.191402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.232059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 175/349] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:40.233251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981221-bc5d-4357-bab4-e6e471e04aba None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.272806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:40.288898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:40.345952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130e7d1-7caf-4a17-90b5-b5bd908dbeb4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.387418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981221-bc5d-4357-bab4-e6e471e04aba None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.397308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.398097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.398383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.398654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.445025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.446563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.447110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.447584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.449053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:40.449610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:40.452107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:40.464796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 176/350] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:40.466258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.483439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:40.484845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:40.560754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:40.580283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:40.613175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565d311-2ff3-4d6c-8398-c3759be9945f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.634664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 175/351] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:40.635661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.655062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.655703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.655984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.656254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.664534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:40.666897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:40.669161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 177/352] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:40.670427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.753524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1b72-3457-438b-bd5b-492826716fe3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.794805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822d135-1c2a-4a17-9656-71ac00aa33bd None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.798237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceb0d9-fa82-4f33-8485-ac859c0d72c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.798879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb0d9-fa82-4f33-8485-ac859c0d72c9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.799158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb0d9-fa82-4f33-8485-ac859c0d72c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.799434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb0d9-fa82-4f33-8485-ac859c0d72c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.816462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceb0d9-fa82-4f33-8485-ac859c0d72c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a0ceb0d9-fa82-4f33-8485-ac859c0d72c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:40.821367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:40.822249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:40.823028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 176/353] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:40.823705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c6f9f-f62c-4284-b291-e2acbd4e65fc None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.824521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c6f9f-f62c-4284-b291-e2acbd4e65fc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.824521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c6f9f-f62c-4284-b291-e2acbd4e65fc None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.829343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 177/354] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:40.829343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c6f9f-f62c-4284-b291-e2acbd4e65fc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.843107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:40.843762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:40.844043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:40.844315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] PATH_INFO: `/v3/users/005dedb64fb8476b849ca5b24feaf920` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:40.845523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] RBAC: Authorizing `identity:get_user(user_id=005dedb64fb8476b849ca5b24feaf920)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:40.846987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:40.848731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 178/355] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/users/005dedb64fb8476b849ca5b24feaf920 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:40.849353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:40.856551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:40.864656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:40.956381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c6f9f-f62c-4284-b291-e2acbd4e65fc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:40.972780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bd58c-6b74-43eb-8f88-3aec017bb98f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.001045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.001788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.002070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.002343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.003521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.005039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.006693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 178/356] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:41.009342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.016766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea158e3-fd9c-4a6f-97ff-5235ea9e60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.017613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea158e3-fd9c-4a6f-97ff-5235ea9e60bf None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.017892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea158e3-fd9c-4a6f-97ff-5235ea9e60bf None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.018164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea158e3-fd9c-4a6f-97ff-5235ea9e60bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.019336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea158e3-fd9c-4a6f-97ff-5235ea9e60bf None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=6d75a1525d11453faac47809e5dc2f01) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:41.019336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea158e3-fd9c-4a6f-97ff-5235ea9e60bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.020741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea158e3-fd9c-4a6f-97ff-5235ea9e60bf None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.027866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 179/357] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:40 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:41.028882 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694157f-ce50-4573-9393-4579f94c356c None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.029258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694157f-ce50-4573-9393-4579f94c356c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.030414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694157f-ce50-4573-9393-4579f94c356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.053681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8694157f-ce50-4573-9393-4579f94c356c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:41.055346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8694157f-ce50-4573-9393-4579f94c356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:41.122878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:41.148396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840276-a557-46ee-9a9c-732375978507 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.162346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 180/358] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:41.163228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694157f-ce50-4573-9393-4579f94c356c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.172818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:41.193314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485fb19-ae39-4200-a5aa-c25870bbe2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.194833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485fb19-ae39-4200-a5aa-c25870bbe2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.195904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485fb19-ae39-4200-a5aa-c25870bbe2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.196374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485fb19-ae39-4200-a5aa-c25870bbe2b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.197406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485fb19-ae39-4200-a5aa-c25870bbe2b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:41.197717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485fb19-ae39-4200-a5aa-c25870bbe2b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.199361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485fb19-ae39-4200-a5aa-c25870bbe2b4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.208992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 179/359] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:41.210273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d60a-25e0-481a-80bc-d24896d558f0 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.211265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d60a-25e0-481a-80bc-d24896d558f0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.211265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d60a-25e0-481a-80bc-d24896d558f0 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.212172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 180/360] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:41.213859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d60a-25e0-481a-80bc-d24896d558f0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.290292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694157f-ce50-4573-9393-4579f94c356c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.335669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.336234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d60a-25e0-481a-80bc-d24896d558f0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.336342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.336592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.336818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.337634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] RBAC: Adding query filter params (role.id=97e9dc3569a94caeb432886a8e5e708c, user.id=005dedb64fb8476b849ca5b24feaf920, scope.system=all) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:41.337941 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.339701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.361304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 181/361] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/role_assignments?user.id=005dedb64fb8476b849ca5b24feaf920&scope.system=all&role.id=97e9dc3569a94caeb432886a8e5e708c => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:41.362170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.386390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.387440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.388004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.388597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.390109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.392032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.394511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 181/362] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:41.395669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.414817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:41.442623 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:41.487341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c092e-3259-4fc1-9213-be5e1297d810 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.520505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5a8f0-c8c3-4215-9c30-ffa5f15b91ef None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.526248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc9bff-c316-4d91-8bea-463e94edbe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.526900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc9bff-c316-4d91-8bea-463e94edbe03 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.527180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc9bff-c316-4d91-8bea-463e94edbe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.527451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc9bff-c316-4d91-8bea-463e94edbe03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.541059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc9bff-c316-4d91-8bea-463e94edbe03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-9ebc9bff-c316-4d91-8bea-463e94edbe03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:41.546352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:41.547240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:41.548039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:41.549101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 182/363] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:41.556361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207229-7b9e-483d-abd3-3db21d38ce94 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.556943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207229-7b9e-483d-abd3-3db21d38ce94 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.556943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207229-7b9e-483d-abd3-3db21d38ce94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.568249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.568864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.569145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.569472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.582150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5c207229-7b9e-483d-abd3-3db21d38ce94 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:41.583685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5c207229-7b9e-483d-abd3-3db21d38ce94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:41.661996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.664785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.681915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 183/364] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:41.683142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c207229-7b9e-483d-abd3-3db21d38ce94 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.684379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:41.757923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:41.803129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 182/365] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:41.803907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.836335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c207229-7b9e-483d-abd3-3db21d38ce94 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.890001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.890364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.890643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.891613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.891795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=6d75a1525d11453faac47809e5dc2f01) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:41.892085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.895307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.903743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 184/366] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=807306af822540d7986dfb93d0b3f344 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:41.904450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:41.933212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bfb07-e622-4ded-b7df-e417f5f50a5b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:41.951393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:41.973973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:41.974746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:41.974980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:41.975256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:41.975957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:41.976299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:41.977782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:41.987118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 183/367] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:41.987982 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.044750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bd888-d3fc-40b8-957e-a52c8f8f2fd1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.067100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:42.090861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.091885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.092367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.092955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.099475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8cea8-e7b5-4d2d-bc24-1fb96a8e685c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.106879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:42.112637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:42.113561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:42.114345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:42.117633 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 185/368] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:42.118674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.147671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.148967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.150028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.150527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.152124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:42.154428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:42.156865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 184/369] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:41 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:42.158139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.220120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:42.265953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aca4e8-503d-43d8-ac3e-4598749e988c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.290758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540233-0cf3-4101-8af5-9878b3661bea None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.318193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.318767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.319021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.319296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.335815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.336848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.337136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.337721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.338403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:42.338709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:42.340857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:42.352454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 185/370] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:42.353862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.381195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:42.413151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:42.415628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:42.474864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d1b9f-079c-46ca-959a-6a351d80a63b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.485057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:42.528602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.529254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.529714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.529988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.533961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:42.535724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:42.537985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 186/371] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:42.540061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.551204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 186/372] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:42.552538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.657033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:42.668861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023e783-00df-4028-bace-caedabcb81e9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.683868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ed988-ae56-4384-a325-b300cd3c388a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.704978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.705641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.705920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.706193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.722227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.723285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.723859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.724432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.725448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:42.725961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:42.728683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:42.735092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 187/373] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:42.736259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.757542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:42.787820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:42.790169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:42.857648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a15c-745c-4dad-a91c-5dd3c1251471 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:42.895809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:42.896881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:42.897465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:42.898597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:42.900181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:42.902014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:42.902825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 187/374] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:42.904541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 188/375] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:42.908646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.911981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:42.930819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:42.994603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:43.022250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ab55e-7426-4328-bde2-db83ff4a53e3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:43.071003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:43.072124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.073428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.074553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] PATH_INFO: `/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.077797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] RBAC: Authorizing `identity:get_project(project_id=bd2efc376f4648d0a2b40e0c73cde3c5)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:43.078346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ce4c-6511-4186-a4b3-5bf66015c45e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:43.081750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:43.093556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 188/376] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:43.097676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:43.144674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:43.145924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.146803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.147295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.149257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:43.151443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:43.153591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 189/377] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:42 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:43.162345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:43.298164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddfcc1-a70b-4b1f-9591-92515381cf9a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:43.308687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:43.337821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb9e72-ccdb-4631-af70-bfda4d8a3c4a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:43.370897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:43.372187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.372464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.372732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] PATH_INFO: `/v3/groups/94244b5d6c244b7e980e50a7ff9650db` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.380985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] RBAC: Authorizing `identity:get_group(group_id=94244b5d6c244b7e980e50a7ff9650db)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:43.383092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:43.389312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 189/378] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:43 2022] GET /identity/v3/groups/94244b5d6c244b7e980e50a7ff9650db => generated 229 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:43.391423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f921-8220-4f6c-abc6-571cc79d6c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:43.392053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f921-8220-4f6c-abc6-571cc79d6c3e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.392327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f921-8220-4f6c-abc6-571cc79d6c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.392595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f921-8220-4f6c-abc6-571cc79d6c3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.393353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f921-8220-4f6c-abc6-571cc79d6c3e None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:43.393663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f921-8220-4f6c-abc6-571cc79d6c3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:43.395165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f921-8220-4f6c-abc6-571cc79d6c3e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:43.398458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:43.408030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:43.415407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 190/379] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:43 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=807306af822540d7986dfb93d0b3f344 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:43.457121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f36977-d2a0-47fc-9132-9542e8d50567 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.457510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f36977-d2a0-47fc-9132-9542e8d50567 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.457707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f36977-d2a0-47fc-9132-9542e8d50567 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.459343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 191/380] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:43.464861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc43cf-c795-4f3a-b6a8-2d48b3e03548 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.465143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc43cf-c795-4f3a-b6a8-2d48b3e03548 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.465450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc43cf-c795-4f3a-b6a8-2d48b3e03548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.498367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-80fc43cf-c795-4f3a-b6a8-2d48b3e03548 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:43.500001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-80fc43cf-c795-4f3a-b6a8-2d48b3e03548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:43.541200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47eb7-632d-4c39-bd39-17ab62b1d169 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:43.584375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:43.585001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.585291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.585638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.586377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] RBAC: Adding query filter params (group.id=94244b5d6c244b7e980e50a7ff9650db, role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=bd2efc376f4648d0a2b40e0c73cde3c5) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:43.586686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:43.588199 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:43.597301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 190/381] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:43 2022] GET /identity/v3/role_assignments?group.id=94244b5d6c244b7e980e50a7ff9650db&scope.project.id=bd2efc376f4648d0a2b40e0c73cde3c5&role.id=2470da926a0342698ca115a00c86fea3 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:43.643196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 192/382] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:43.677819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:43.785650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:43.826163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272618-843a-4f58-8e77-01b26ba4d676 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:43.868640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:43.869534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:43.869950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:43.870382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:43.886466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:43.891943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:43.892998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:43.894004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:43.898329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 191/383] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:43.901959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc43cf-c795-4f3a-b6a8-2d48b3e03548 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:44.020867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:44.061431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc43cf-c795-4f3a-b6a8-2d48b3e03548 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:44.105973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c26dd6-3a6a-4f00-b430-48236c1ac443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:44.106592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c26dd6-3a6a-4f00-b430-48236c1ac443 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.106863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c26dd6-3a6a-4f00-b430-48236c1ac443 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.107143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c26dd6-3a6a-4f00-b430-48236c1ac443 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.107820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c26dd6-3a6a-4f00-b430-48236c1ac443 None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:44.108126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c26dd6-3a6a-4f00-b430-48236c1ac443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:44.109560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c26dd6-3a6a-4f00-b430-48236c1ac443 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:44.119350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 193/384] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:44.124832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:44.261767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c70547-dddf-4700-a50e-ee6a9daca522 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:44.301051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:44.307390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44744e4-b600-4bb0-95b2-135701e5f5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:44.308026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44744e4-b600-4bb0-95b2-135701e5f5cd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.308297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44744e4-b600-4bb0-95b2-135701e5f5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.308562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44744e4-b600-4bb0-95b2-135701e5f5cd None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.309746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44744e4-b600-4bb0-95b2-135701e5f5cd None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:44.311216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44744e4-b600-4bb0-95b2-135701e5f5cd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:44.311792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652f3d5-a1cb-4593-874d-7b6d83a6eccd None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.312078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652f3d5-a1cb-4593-874d-7b6d83a6eccd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.312357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652f3d5-a1cb-4593-874d-7b6d83a6eccd None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.313082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 192/385] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:44 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:44.313959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 194/386] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:44.318903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37c3c-6418-47f5-8522-84613412c08e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.319406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37c3c-6418-47f5-8522-84613412c08e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.319540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652f3d5-a1cb-4593-874d-7b6d83a6eccd None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:44.320459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37c3c-6418-47f5-8522-84613412c08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.339675 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-23e37c3c-6418-47f5-8522-84613412c08e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:44.340862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-23e37c3c-6418-47f5-8522-84613412c08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:44.438207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 193/387] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:44.468827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e37c3c-6418-47f5-8522-84613412c08e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:44.504912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652f3d5-a1cb-4593-874d-7b6d83a6eccd None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:44.536332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:44.544062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:44.544710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.544984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.545255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.546685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:44.548139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:44.549834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 195/388] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:44 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:44.556193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:44.662616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616d45a-5f38-4800-a2a8-fe47c713ffc3 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:44.699791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:44.700455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.700779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.701060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.703022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d75a1525d11453faac47809e5dc2f01, group_id=89b35c0e46ae4e30bdfd6064445c85ae, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:44.704462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:44.710633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e37c3c-6418-47f5-8522-84613412c08e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:44.726828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 196/389] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 10 12:18:44 2022] PUT /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:18:44.760489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:44.774800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:44.775460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.775741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.776013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.797950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:44.802783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:44.803671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:44.805430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 194/390] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:44.810640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:44.894484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:44.950695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297be54-f5ae-401f-9dcd-e8a48d9301e1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:44.996831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:44.997496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:44.997777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:44.998047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:44.998809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:44.999115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:45.000519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:45.013037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 197/391] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:45.018937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:45.145929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639f4fd-4611-48b7-8d98-af4f8c0ad938 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:45.182078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:45.182712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:45.182987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:45.183265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:45.184385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:45.185885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:45.187749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 195/392] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:45 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:45.193215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:45.315163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70597e5a-ef94-4c20-a2a7-1449182b9546 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:45.338192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:45.350907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa37a04-9439-43d0-95fb-3492778b9f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:45.352265 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa37a04-9439-43d0-95fb-3492778b9f7f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:45.352654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa37a04-9439-43d0-95fb-3492778b9f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:45.353309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa37a04-9439-43d0-95fb-3492778b9f7f None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:45.354777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa37a04-9439-43d0-95fb-3492778b9f7f None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:45.356496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa37a04-9439-43d0-95fb-3492778b9f7f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:45.358653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 198/393] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:45 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:45.364105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:45.457898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:45.511904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed17cf-7218-4968-9287-04ad6df5440d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:45.560718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76683af5-e619-4538-9f06-6de8559b5354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:45.561434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76683af5-e619-4538-9f06-6de8559b5354 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:45.561719 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76683af5-e619-4538-9f06-6de8559b5354 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:45.562004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76683af5-e619-4538-9f06-6de8559b5354 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:45.564182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76683af5-e619-4538-9f06-6de8559b5354 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d75a1525d11453faac47809e5dc2f01, group_id=89b35c0e46ae4e30bdfd6064445c85ae, role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:45.565684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76683af5-e619-4538-9f06-6de8559b5354 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:45.594429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 196/394] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 10 12:18:45 2022] PUT /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/807306af822540d7986dfb93d0b3f344 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:45.817665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec67987-47b8-421d-a85b-7d29ffd6c3d4 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:45.817859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec67987-47b8-421d-a85b-7d29ffd6c3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:45.818156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec67987-47b8-421d-a85b-7d29ffd6c3d4 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:45.819941 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 199/395] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:45.832573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21185d49-ea78-4da4-8c7b-f7abb9e3bc5a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:45.832863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21185d49-ea78-4da4-8c7b-f7abb9e3bc5a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:45.833145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21185d49-ea78-4da4-8c7b-f7abb9e3bc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:45.858576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-21185d49-ea78-4da4-8c7b-f7abb9e3bc5a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:45.859798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-21185d49-ea78-4da4-8c7b-f7abb9e3bc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:45.931391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 197/396] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:45.953885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec67987-47b8-421d-a85b-7d29ffd6c3d4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.082651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec67987-47b8-421d-a85b-7d29ffd6c3d4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.106347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9dad-31f0-41ac-a17b-61aa4ac65ec9 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.106636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9dad-31f0-41ac-a17b-61aa4ac65ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.106855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9dad-31f0-41ac-a17b-61aa4ac65ec9 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.108453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 198/397] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:46.122657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.122981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb14c71-1263-4fa4-a82c-86bb51d41bba None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.123194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb14c71-1263-4fa4-a82c-86bb51d41bba None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.123475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb14c71-1263-4fa4-a82c-86bb51d41bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.124253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.124719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.125201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.137884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:46.143272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:46.144139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:46.144967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:46.147688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 200/398] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:46.155064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.165312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6eb14c71-1263-4fa4-a82c-86bb51d41bba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:46.166528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6eb14c71-1263-4fa4-a82c-86bb51d41bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:46.210173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:46.237415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:46.268944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657f478-ab1c-4f40-a4fb-b8798499d708 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.276248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 199/399] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:46.276248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0eee1-79dc-4c97-b161-66b42f2e3b19 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.276248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0eee1-79dc-4c97-b161-66b42f2e3b19 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.276248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0eee1-79dc-4c97-b161-66b42f2e3b19 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.277680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 200/400] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:46.279120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f6aa4-3f1c-4b65-b116-114883f7f3d3 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.279410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f6aa4-3f1c-4b65-b116-114883f7f3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.279687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f6aa4-3f1c-4b65-b116-114883f7f3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.300667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-582f6aa4-3f1c-4b65-b116-114883f7f3d3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:46.302510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-582f6aa4-3f1c-4b65-b116-114883f7f3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:46.329510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.330492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.331027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.331589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.332541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:46.333102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:46.334862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:46.345349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 201/401] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:46.351902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.391467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 201/402] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:46.397619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f6aa4-3f1c-4b65-b116-114883f7f3d3 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.471944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f9e08-8e63-4f38-a023-4b331dddb57c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.486517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:46.510522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.511456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.511987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.512464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.524093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:46.529707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:46.530562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:46.531348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:46.534013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 202/403] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:46.540033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.540154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f6aa4-3f1c-4b65-b116-114883f7f3d3 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.548513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:46.590629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.591200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.591480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.591809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.594785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:46.594785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:46.600643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 202/404] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:46.600643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.668700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824065a-e964-48b1-ad53-15fc3ab2e64a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.710879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.711519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.711803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.712079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.726169 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:46.726670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88ecb0-b8f3-4234-9230-8e06b2c6608f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:46.733129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:46.734038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:46.734810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:46.739635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 203/405] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:46.740490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.768667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.769657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.770179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.770662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.771610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:46.772159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:46.773837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:46.782635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 203/406] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:46.783053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:46.783879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.824486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:46.876635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eee91b-ad11-4cec-a529-768a2281e982 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.904061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116778b-ee5d-4ddb-b580-a8db75fd08e9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:46.913175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.914669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.915210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.915673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.917087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:46.918812 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:46.925799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 204/407] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:46.926861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:46.948429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:46.949049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:46.949357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:46.949638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:46.950315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:46.950624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:46.952090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:46.968647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 204/408] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:46.969644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.035592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:47.041129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf1bbb-9f5e-490d-8524-99ca548c6c3c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.074615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:47.078155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.079042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.079581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.079751 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50341d4-8304-4739-8a18-1456eed854c9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.080203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] PATH_INFO: `/v3/groups/94244b5d6c244b7e980e50a7ff9650db` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.081576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] RBAC: Authorizing `identity:get_group(group_id=94244b5d6c244b7e980e50a7ff9650db)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:47.083204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:47.085308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 205/409] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:46 2022] GET /identity/v3/groups/94244b5d6c244b7e980e50a7ff9650db => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:47.086343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.128096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.129286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.130020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.130360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.132762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f691be5ba412783c3ac8c173ed84a, group_id=89b35c0e46ae4e30bdfd6064445c85ae, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:47.134295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:47.149498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 205/410] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 10 12:18:46 2022] PUT /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:47.150535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.224405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b80b23-d4c5-4faa-b163-3a78eac3a2ef None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.274530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad92229-e14b-4934-b49e-bed9313aed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.275686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad92229-e14b-4934-b49e-bed9313aed54 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.276148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad92229-e14b-4934-b49e-bed9313aed54 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.276568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad92229-e14b-4934-b49e-bed9313aed54 None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.278260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad92229-e14b-4934-b49e-bed9313aed54 None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:47.279921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad92229-e14b-4934-b49e-bed9313aed54 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:47.281838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 206/411] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:47 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:47.283321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80216a7f-af35-4a70-b282-0432fd902796 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.283775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80216a7f-af35-4a70-b282-0432fd902796 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.284211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80216a7f-af35-4a70-b282-0432fd902796 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.290720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 207/412] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:47.291902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80216a7f-af35-4a70-b282-0432fd902796 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.306692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba216dd-c13a-49eb-8ece-ef81eaaca0a5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.333698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:47.349085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee65d0d-d4ed-4f21-9d02-d9c6e1c77060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.349995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65d0d-d4ed-4f21-9d02-d9c6e1c77060 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.350467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65d0d-d4ed-4f21-9d02-d9c6e1c77060 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.350909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee65d0d-d4ed-4f21-9d02-d9c6e1c77060 None admin] PATH_INFO: `/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.352745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee65d0d-d4ed-4f21-9d02-d9c6e1c77060 None admin] RBAC: Authorizing `identity:get_project(project_id=bd2efc376f4648d0a2b40e0c73cde3c5)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:47.354674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee65d0d-d4ed-4f21-9d02-d9c6e1c77060 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:47.357102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 206/413] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:47 2022] GET /identity/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:47.361292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fbafe-a917-4eb1-908c-00fa6d6ff665 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.361292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fbafe-a917-4eb1-908c-00fa6d6ff665 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.361292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fbafe-a917-4eb1-908c-00fa6d6ff665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.378998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ca1fbafe-a917-4eb1-908c-00fa6d6ff665 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:47.380103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ca1fbafe-a917-4eb1-908c-00fa6d6ff665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:47.404881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:47.424177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80216a7f-af35-4a70-b282-0432fd902796 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.461698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.462425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.462779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.463129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.464330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:47.465120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 207/414] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:47.465976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fbafe-a917-4eb1-908c-00fa6d6ff665 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.473502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:47.475439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 208/415] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:47 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:47.483763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.580971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fbafe-a917-4eb1-908c-00fa6d6ff665 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.616780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.617075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cb9bd-77ac-4dd1-88cb-483ef13eeebb None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.617893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.618381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.618817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] PATH_INFO: `/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5/groups/94244b5d6c244b7e980e50a7ff9650db/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.619462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:47.621707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd2efc376f4648d0a2b40e0c73cde3c5, group_id=94244b5d6c244b7e980e50a7ff9650db, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:47.623647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:47.638719 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 208/416] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 10 12:18:47 2022] PUT /identity/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5/groups/94244b5d6c244b7e980e50a7ff9650db/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:18:47.639811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.681810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.684299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.684742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.685683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.688192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:47.692489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f691be5ba412783c3ac8c173ed84a, group_id=89b35c0e46ae4e30bdfd6064445c85ae, role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:47.694590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:47.724701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 209/417] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 10 12:18:47 2022] PUT /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a/groups/89b35c0e46ae4e30bdfd6064445c85ae/roles/807306af822540d7986dfb93d0b3f344 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:18:47.805801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb819f-9cb8-4d00-a9d7-ccba068529d6 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:47.844144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191be9c-559c-41c7-8bf1-c72b9238a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:47.844808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191be9c-559c-41c7-8bf1-c72b9238a382 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.845088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191be9c-559c-41c7-8bf1-c72b9238a382 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.845394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191be9c-559c-41c7-8bf1-c72b9238a382 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.859380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191be9c-559c-41c7-8bf1-c72b9238a382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f191be9c-559c-41c7-8bf1-c72b9238a382 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:47.864809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:47.865812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:47.867243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 209/418] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:47.874832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:47.907124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2113e-ded4-444d-81f2-b910308908bd None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.907306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2113e-ded4-444d-81f2-b910308908bd None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.907595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2113e-ded4-444d-81f2-b910308908bd None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.908961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 210/419] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:18:47.914789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3e161-6cf2-4a27-91db-3cf01e43bf80 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:47.915086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3e161-6cf2-4a27-91db-3cf01e43bf80 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:47.915333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3e161-6cf2-4a27-91db-3cf01e43bf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:47.944171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ffd3e161-6cf2-4a27-91db-3cf01e43bf80 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:47.946107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ffd3e161-6cf2-4a27-91db-3cf01e43bf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:47.999890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c42e8d-6b16-4e0e-8b8e-4613ca2e93e9 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:48.029066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 211/420] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:48.036897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:48.037541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:48.037820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:48.038091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:48.038801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:48.039101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:48.040475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:48.049967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 210/421] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:48.056133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:48.056278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3e161-6cf2-4a27-91db-3cf01e43bf80 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:48.065328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:48.098866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:48.163722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3e161-6cf2-4a27-91db-3cf01e43bf80 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:48.199484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:48.200474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:48.200949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:48.201461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:48.212261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:48.217990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:48.218880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:48.219685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:48.222238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 212/422] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:48.228493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:48.254763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b4f44-290c-4e93-8cbe-82e19400e66d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:48.299063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:48.300044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:48.300536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:48.301163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:48.353760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:48.383994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:48.386624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:48.390962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f72fa-85f0-465b-bdec-52cfb3dd6d74 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:48.417678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:48.479156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:48.480455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:48.480916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:48.481402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:48.482269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:48.482877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:48.484466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:48.511554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 213/423] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:48.518282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:48.521198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 211/424] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:48.527933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:48.723176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6cb4d-6154-450c-9ea3-27c0e46aed84 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:48.728646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60b050-f3ef-44ce-88a3-b0bb2cef9406 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:48.762692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:48.775759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:48.776863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:48.779984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:48.780475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:48.781903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:48.783700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:48.786578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 212/425] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:48 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:48.793501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:48.794330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:48.814345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:48.815296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:48.815795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:48.816275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:48.937644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:48.940238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:48.969929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a55f1-d549-46ee-b92d-0f2e73425337 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:49.042340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:49.043312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.043829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.044454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.046991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:49.049998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:49.052467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 213/426] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:48 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:49.059659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:49.132009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 214/427] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:49.138062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:49.161047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:49.194000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:49.224605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce183-1a32-4330-b3b2-bb8f5d091adf None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:49.291192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:49.291834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.292065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.292330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.293111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=6d75a1525d11453faac47809e5dc2f01) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:49.293385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:49.294832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:49.307357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 214/428] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:49 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 611 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:49.333808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479b7f3-7fd7-48ea-8e86-12776f7a8c18 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:49.395489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116f0ec-c0ea-47f7-a65a-052b4547b6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:49.396142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f0ec-c0ea-47f7-a65a-052b4547b6cd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.396419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f0ec-c0ea-47f7-a65a-052b4547b6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.396691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f0ec-c0ea-47f7-a65a-052b4547b6cd None admin] PATH_INFO: `/v3/projects/6d75a1525d11453faac47809e5dc2f01` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.397894 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116f0ec-c0ea-47f7-a65a-052b4547b6cd None admin] RBAC: Authorizing `identity:get_project(project_id=6d75a1525d11453faac47809e5dc2f01)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:49.399335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116f0ec-c0ea-47f7-a65a-052b4547b6cd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:49.400966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 215/429] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:49 2022] GET /identity/v3/projects/6d75a1525d11453faac47809e5dc2f01 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:49.408858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:49.547122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e3124-1f3e-46c1-9ec7-e357e015e7ac None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:49.585772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:49.587440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:49.588488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.588592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.588829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.589992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:49.597474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:49.597722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9681c-8bcc-4181-99a0-cbbcf187021a None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.598158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9681c-8bcc-4181-99a0-cbbcf187021a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.598243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9681c-8bcc-4181-99a0-cbbcf187021a None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.599860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 216/430] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:49.601385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 215/431] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:49 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:49.606689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164363c-9dfe-4eb9-9d0c-1b723be26b74 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.606907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164363c-9dfe-4eb9-9d0c-1b723be26b74 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.607179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164363c-9dfe-4eb9-9d0c-1b723be26b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.608251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:49.626060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b164363c-9dfe-4eb9-9d0c-1b723be26b74 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:49.628103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b164363c-9dfe-4eb9-9d0c-1b723be26b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:49.716152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 217/432] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:49.742425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164363c-9dfe-4eb9-9d0c-1b723be26b74 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:49.751459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad39fe-e8fa-41b9-ae29-da705b473a38 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:49.775808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:49.792886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:49.793870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.793994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.794224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.794958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=6d75a1525d11453faac47809e5dc2f01) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:49.795262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:49.796794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:49.803631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 216/433] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:49 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=6d75a1525d11453faac47809e5dc2f01&role.id=807306af822540d7986dfb93d0b3f344 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:49.872838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164363c-9dfe-4eb9-9d0c-1b723be26b74 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:49.911810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86db81-680d-4136-9078-c1c839f066e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:49.912781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86db81-680d-4136-9078-c1c839f066e7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:49.913216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86db81-680d-4136-9078-c1c839f066e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:49.914206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86db81-680d-4136-9078-c1c839f066e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:49.926477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86db81-680d-4136-9078-c1c839f066e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-2c86db81-680d-4136-9078-c1c839f066e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:49.932706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:49.933863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 12:18:49.934859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:49.937323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 218/434] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:49.943725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.017185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.058436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef862a-bcb8-47ec-9944-0f73cb221efa None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.058645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef862a-bcb8-47ec-9944-0f73cb221efa None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.058929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef862a-bcb8-47ec-9944-0f73cb221efa None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.060552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 219/435] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:50.066046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc8d3b-7beb-4343-a210-1b3423df12b0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.066837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da649d4-6d86-4c65-aa3d-9ca0fa2f5409 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.067125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da649d4-6d86-4c65-aa3d-9ca0fa2f5409 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.067396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da649d4-6d86-4c65-aa3d-9ca0fa2f5409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.089538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2da649d4-6d86-4c65-aa3d-9ca0fa2f5409 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:50.090748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2da649d4-6d86-4c65-aa3d-9ca0fa2f5409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:50.107501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.108524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.108816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.109100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.109902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] RBAC: Adding query filter params (name=member) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:50.110210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:50.111600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:50.121659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 217/436] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:18:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:50.127190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.172528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 220/437] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:50.195222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da649d4-6d86-4c65-aa3d-9ca0fa2f5409 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.215599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.251910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7feb1-9bdd-4a3c-86a4-3e1304220776 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.275015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.306138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9b573-85ef-4dcc-ad79-f8fd77952bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.306797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9b573-85ef-4dcc-ad79-f8fd77952bfa None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.307018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9b573-85ef-4dcc-ad79-f8fd77952bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.307292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9b573-85ef-4dcc-ad79-f8fd77952bfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.313551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da649d4-6d86-4c65-aa3d-9ca0fa2f5409 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.348836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a615a3-df3b-4370-b49a-79b76d9c3eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.349510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a615a3-df3b-4370-b49a-79b76d9c3eea None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.349774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a615a3-df3b-4370-b49a-79b76d9c3eea None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.350042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a615a3-df3b-4370-b49a-79b76d9c3eea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.361911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a615a3-df3b-4370-b49a-79b76d9c3eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-d8a615a3-df3b-4370-b49a-79b76d9c3eea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:50.368067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:50.369441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 221/438] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:50.370328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383cab7-2be9-4bae-96c8-d912ad4eaf4c None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.372621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383cab7-2be9-4bae-96c8-d912ad4eaf4c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.372621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383cab7-2be9-4bae-96c8-d912ad4eaf4c None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.372621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 222/439] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:50.374043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383cab7-2be9-4bae-96c8-d912ad4eaf4c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.399436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9b573-85ef-4dcc-ad79-f8fd77952bfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:50.400922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9b573-85ef-4dcc-ad79-f8fd77952bfa None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:50.456679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.481828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383cab7-2be9-4bae-96c8-d912ad4eaf4c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.497209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 218/440] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:50.498568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b238a-e2a4-4bdb-93b5-439a695307ae None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.498930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b238a-e2a4-4bdb-93b5-439a695307ae None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.499268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b238a-e2a4-4bdb-93b5-439a695307ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.516557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fb4b238a-e2a4-4bdb-93b5-439a695307ae None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:50.517766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fb4b238a-e2a4-4bdb-93b5-439a695307ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:50.519837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.520441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.520722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.521000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.521695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:50.522001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:50.523919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:50.531891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 223/441] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:50.532555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.532668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.589517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 219/442] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:50.590368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b238a-e2a4-4bdb-93b5-439a695307ae None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.649232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e85e8-da29-4ece-a86b-33ba6c0ef799 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.682542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.683155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.683437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.683702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.684832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:50.686307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:50.687067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b238a-e2a4-4bdb-93b5-439a695307ae None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.687915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 224/443] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:50.688658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.697538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.725242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.725875 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.726153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.726426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.744424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.787153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b467ce6-2b32-42f8-b824-e9eaad750678 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.792987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:50.794493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:50.818711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.819328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.820113 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.820396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.830172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:50.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:50.836884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 225/444] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:50.837849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.895483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 220/445] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:50.896222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.902078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.936157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ff332-8cb7-4124-afd9-9dccbd593725 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:50.943212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:50.970168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:50.970790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:50.971068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:50.971342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:50.972471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:50.973956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:50.975526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 226/446] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:50.976283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:50.993473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a35aa-f7e1-4e48-919a-332cf72a2d6d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.026540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.027156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.027437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.027718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.028408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:51.028709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.030106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.038379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 221/447] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:51.039030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:51.075170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b114412-50cc-4117-8afe-8d42e2ef1477 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.106973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.107584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.107857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.108125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] PATH_INFO: `/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.109223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] RBAC: Authorizing `identity:get_project(project_id=bd2efc376f4648d0a2b40e0c73cde3c5)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.110715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.112308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 227/448] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:50 2022] GET /identity/v3/projects/bd2efc376f4648d0a2b40e0c73cde3c5 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:51.113058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:51.115496 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:51.141893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07669a1c-cf27-4356-b614-394a35742737 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.154259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:51.175386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.175988 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.176267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.176543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.177278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=ae3c9f4504f84e25bc89e8c30397aedc, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:51.177600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.179541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.185869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 222/449] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:51 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=ae3c9f4504f84e25bc89e8c30397aedc => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:51.186635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:51.216479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041d74a-d41a-4dc1-99c9-d2428d3273f8 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.259761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.260664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.261083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.261547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.309183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5aca06-a81e-4bed-8e8f-64d83e465974 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.340296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.341821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.343737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:51.346199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.346843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.347120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.347393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] PATH_INFO: `/v3/groups/94244b5d6c244b7e980e50a7ff9650db` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.348591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] RBAC: Authorizing `identity:get_group(group_id=94244b5d6c244b7e980e50a7ff9650db)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.350303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.352174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 223/450] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:51 2022] GET /identity/v3/groups/94244b5d6c244b7e980e50a7ff9650db => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:51.358298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:51.418382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:51.455470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 228/451] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:51.461569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:51.483376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255f86e-a66a-4089-a099-004320a46446 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.523102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.524122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.524122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.524305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.527299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] RBAC: Adding query filter params (group.id=94244b5d6c244b7e980e50a7ff9650db, role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=bd2efc376f4648d0a2b40e0c73cde3c5) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:51.527299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.527299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.532683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 224/452] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:51 2022] GET /identity/v3/role_assignments?group.id=94244b5d6c244b7e980e50a7ff9650db&scope.project.id=bd2efc376f4648d0a2b40e0c73cde3c5&role.id=2470da926a0342698ca115a00c86fea3 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:51.582555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bb2d6-f80a-471d-aac8-0ddb251a83f8 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.584297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:51.621519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.622159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.622434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.622707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] PATH_INFO: `/v3/projects/8e0f691be5ba412783c3ac8c173ed84a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.623849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] RBAC: Authorizing `identity:get_project(project_id=8e0f691be5ba412783c3ac8c173ed84a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.625361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.627023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 229/453] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:18:51 2022] GET /identity/v3/projects/8e0f691be5ba412783c3ac8c173ed84a => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:18:51.632651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:51.734837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350cfa2-a5b2-40e2-8efc-2a3fba81c7b4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.743538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:51.767482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3d382-88c2-494d-aa33-9390d10c845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.768110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3d382-88c2-494d-aa33-9390d10c845b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.768388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3d382-88c2-494d-aa33-9390d10c845b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.768663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3d382-88c2-494d-aa33-9390d10c845b None admin] PATH_INFO: `/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.769806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3d382-88c2-494d-aa33-9390d10c845b None admin] RBAC: Authorizing `identity:get_group(group_id=89b35c0e46ae4e30bdfd6064445c85ae)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.771245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3d382-88c2-494d-aa33-9390d10c845b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.773326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 225/454] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 10 12:18:51 2022] GET /identity/v3/groups/89b35c0e46ae4e30bdfd6064445c85ae => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:51.779111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:51.875196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce61763-33bf-4529-aca6-0e965693219c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:51.907802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f39930-8cf1-4249-9a41-e42222c4b80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:51.908388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39930-8cf1-4249-9a41-e42222c4b80e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:51.908666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39930-8cf1-4249-9a41-e42222c4b80e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:51.908934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39930-8cf1-4249-9a41-e42222c4b80e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:51.909742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f39930-8cf1-4249-9a41-e42222c4b80e None admin] RBAC: Adding query filter params (group.id=89b35c0e46ae4e30bdfd6064445c85ae, role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=8e0f691be5ba412783c3ac8c173ed84a) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:51.910049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f39930-8cf1-4249-9a41-e42222c4b80e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:51.911466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f39930-8cf1-4249-9a41-e42222c4b80e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:51.917632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 230/455] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 10 12:18:51 2022] GET /identity/v3/role_assignments?group.id=89b35c0e46ae4e30bdfd6064445c85ae&scope.project.id=8e0f691be5ba412783c3ac8c173ed84a&role.id=807306af822540d7986dfb93d0b3f344 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:54.339630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e598-4494-41e1-926f-a5c77e53eecf None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.340185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e598-4494-41e1-926f-a5c77e53eecf None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.340454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e598-4494-41e1-926f-a5c77e53eecf None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.342201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 226/456] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:54.347507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b41452-92d4-4661-b33c-c4c7af769229 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.347959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b41452-92d4-4661-b33c-c4c7af769229 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.348048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b41452-92d4-4661-b33c-c4c7af769229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.372580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d4b41452-92d4-4661-b33c-c4c7af769229 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:54.373936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d4b41452-92d4-4661-b33c-c4c7af769229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:54.402957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:54.444327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 231/457] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:54.486491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9e598-4494-41e1-926f-a5c77e53eecf None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:54.634750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9e598-4494-41e1-926f-a5c77e53eecf None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:54.681256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982231a-6992-4cc4-ba0d-0a09a18db18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:54.681917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982231a-6992-4cc4-ba0d-0a09a18db18d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.682201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982231a-6992-4cc4-ba0d-0a09a18db18d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.682477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982231a-6992-4cc4-ba0d-0a09a18db18d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.700935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982231a-6992-4cc4-ba0d-0a09a18db18d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:54.702443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982231a-6992-4cc4-ba0d-0a09a18db18d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-3982231a-6992-4cc4-ba0d-0a09a18db18d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:18:54.722122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:54.723247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:18:54.724240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:18:54.724240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:54.724240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:54.724240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 227/458] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:54.730378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b41452-92d4-4661-b33c-c4c7af769229 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:54.731308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c86b32-335e-400b-b9f8-74b6487da4ed None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.731596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c86b32-335e-400b-b9f8-74b6487da4ed None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.731875 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c86b32-335e-400b-b9f8-74b6487da4ed None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.733612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 228/459] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:18:54.739765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb4e5-808a-4243-aadf-9535319ac1c3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.740052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb4e5-808a-4243-aadf-9535319ac1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.740327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb4e5-808a-4243-aadf-9535319ac1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.769305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-897bb4e5-808a-4243-aadf-9535319ac1c3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:54.770800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-897bb4e5-808a-4243-aadf-9535319ac1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:54.800260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:54.858043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 229/460] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:54.858467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b41452-92d4-4661-b33c-c4c7af769229 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:54.859244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fe57a-e16a-4f28-b5ec-183e2c14f247 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.859531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fe57a-e16a-4f28-b5ec-183e2c14f247 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.859808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fe57a-e16a-4f28-b5ec-183e2c14f247 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.862507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 230/461] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:54.868872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6fec0-3f5b-4028-9c00-e148f4c3f302 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.869154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6fec0-3f5b-4028-9c00-e148f4c3f302 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.869459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6fec0-3f5b-4028-9c00-e148f4c3f302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.897993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e1e6fec0-3f5b-4028-9c00-e148f4c3f302 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:54.899187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e1e6fec0-3f5b-4028-9c00-e148f4c3f302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:54.911919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:54.912542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:54.912824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:54.913098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:54.913925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:54.914338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:54.915740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:54.928768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 232/462] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:54.934361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:54.961899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:54.970488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 231/463] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:54.976989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6fec0-3f5b-4028-9c00-e148f4c3f302 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:55.048394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79607-280c-4de3-9423-53a2eb51429d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:55.095584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:55.096626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.097192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.097508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.107789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:55.115118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:55.117829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:55.124165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6fec0-3f5b-4028-9c00-e148f4c3f302 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:18:55.135584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:55.137383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:18:55.138637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:18:55.138637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:55.138637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:55.138637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 233/464] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:55.142516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:55.172542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:55.173493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.173784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.174059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.174905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:55.177041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:55.207050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 232/465] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:54 2022] POST /identity/v3/users => generated 246 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:55.207896 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:55.258132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:55.289844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e2eb7-98c0-46df-9c31-eb03d7f9a889 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:55.337847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ddc4a-20cc-4cd9-a364-a1da75dcf990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:55.338594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ddc4a-20cc-4cd9-a364-a1da75dcf990 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.338950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ddc4a-20cc-4cd9-a364-a1da75dcf990 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.339295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ddc4a-20cc-4cd9-a364-a1da75dcf990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.351349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ddc4a-20cc-4cd9-a364-a1da75dcf990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:55.353196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ddc4a-20cc-4cd9-a364-a1da75dcf990 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:55.368195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5fbd6-3b6a-4480-b631-5c914d9d440f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-bd3ddc4a-20cc-4cd9-a364-a1da75dcf990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:18:55.370103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:55.371169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:18:55.372154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:18:55.372154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:55.372154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:55.377383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 234/466] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:55.378847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6ad0-2709-4bfc-a508-7c26492ac061 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.379280 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6ad0-2709-4bfc-a508-7c26492ac061 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.379696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6ad0-2709-4bfc-a508-7c26492ac061 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.383992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 235/467] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:55.384950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a6ad0-2709-4bfc-a508-7c26492ac061 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:55.410041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36968a-e135-44db-9e13-4ff33fd76813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:55.410663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36968a-e135-44db-9e13-4ff33fd76813 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.412121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36968a-e135-44db-9e13-4ff33fd76813 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.412472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36968a-e135-44db-9e13-4ff33fd76813 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.413342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36968a-e135-44db-9e13-4ff33fd76813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:55.413588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36968a-e135-44db-9e13-4ff33fd76813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:55.414935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36968a-e135-44db-9e13-4ff33fd76813 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:55.434079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 233/468] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:55.435392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7a7d-f7cf-4a28-817a-d966d5eecdaa None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.435680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7a7d-f7cf-4a28-817a-d966d5eecdaa None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.435961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7a7d-f7cf-4a28-817a-d966d5eecdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.443572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:55.457124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-dd9e7a7d-f7cf-4a28-817a-d966d5eecdaa None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:55.458483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-dd9e7a7d-f7cf-4a28-817a-d966d5eecdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:55.508491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a6ad0-2709-4bfc-a508-7c26492ac061 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:55.539764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:55.547764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 234/469] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:55.548646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:55.548792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e7a7d-f7cf-4a28-817a-d966d5eecdaa None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:55.549448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.549798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.550146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.550847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:55.551218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:55.552694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:55.570117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 236/470] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:55.575895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:55.698133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e7a7d-f7cf-4a28-817a-d966d5eecdaa None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:55.709593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42871095-8252-4cb8-9a15-1a0b407f39e6 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:55.740310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:55.741298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.742299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.742803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.743654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:55.745327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:55.749869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7ed18-58e8-4832-9906-c3f8a92f8ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:55.750473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7ed18-58e8-4832-9906-c3f8a92f8ff6 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:55.750756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7ed18-58e8-4832-9906-c3f8a92f8ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:55.751036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7ed18-58e8-4832-9906-c3f8a92f8ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:55.751747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7ed18-58e8-4832-9906-c3f8a92f8ff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:55.753255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7ed18-58e8-4832-9906-c3f8a92f8ff6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:55.755277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:55.775440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 235/471] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:55 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:55.777028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:55.793854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 237/472] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:55 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:55.962484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976396c5-9763-420b-95d5-f788faff0d28 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:56.008190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8120f-55e2-43ec-9753-82941411e1d3 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.008518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8120f-55e2-43ec-9753-82941411e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.008750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8120f-55e2-43ec-9753-82941411e1d3 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.010307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 238/473] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:18:56.014865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:56.015506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.015781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.016047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.022109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abae53-f35e-4005-b1c8-4f35bc0f6867 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.022410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abae53-f35e-4005-b1c8-4f35bc0f6867 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.022712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abae53-f35e-4005-b1c8-4f35bc0f6867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.029790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:56.031799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:56.046443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-43abae53-f35e-4005-b1c8-4f35bc0f6867 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:18:56.047493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:56.048355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:18:56.049189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:18:56.049189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:56.049189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:56.049421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-43abae53-f35e-4005-b1c8-4f35bc0f6867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:56.049495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 236/474] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:56.057727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:56.102493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:56.115395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:56.130902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 239/475] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:56.155597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abae53-f35e-4005-b1c8-4f35bc0f6867 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:56.200922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100679b9-c199-4cb8-8283-0d4f063a4c3a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:56.238965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:56.239495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.239775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.240051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.240803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:56.241114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:56.242573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:56.255683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 237/476] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:56.261223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:56.271451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abae53-f35e-4005-b1c8-4f35bc0f6867 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:56.312393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:56.313008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.313300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.313584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.314347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:56.315822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:56.326264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.sql.core [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 12:18:56.326264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 12:18:56.326264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [parameters: {'id': '2424f901c73d49be9bba2f27fc0640c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 12:18:56.326264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80064) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result.read() Aug 10 12:18:56.346801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:18:56.347758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application yield resource Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise self.value Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application session.commit() Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 12:18:56.349178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.session.flush() Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application compat.raise_( Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 12:18:56.352467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:18:56.358643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result.read() Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:18:56.360376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [parameters: {'id': '2424f901c73d49be9bba2f27fc0640c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:56.361355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 12:18:56.362376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:56.364004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 240/477] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 10 12:18:56 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:56.364004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:56.377906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df21b8-3aaf-45d0-9d55-9326b4ae0e73 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:56.380248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:56.409119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:56.424136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:56.425021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.425340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.425618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.426264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:56.428156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:56.465969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 238/478] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:18:56 2022] POST /identity/v3/users => generated 249 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:18:56.501847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16fbbb-1caa-4853-b3ce-6930b8f1407a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:56.556309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897712d-118e-47c1-906a-8f3113593011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:56.556970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897712d-118e-47c1-906a-8f3113593011 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.557251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897712d-118e-47c1-906a-8f3113593011 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.557604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897712d-118e-47c1-906a-8f3113593011 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.569972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897712d-118e-47c1-906a-8f3113593011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-e897712d-118e-47c1-906a-8f3113593011 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:56.574786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:56.576872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:56.578856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 241/479] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:18:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:56.582194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:56.694360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be1ad9-0388-4383-8be6-798525183733 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:56.731158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28bc9d-6085-45b1-8fb1-6fc0609f51d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:56.731794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28bc9d-6085-45b1-8fb1-6fc0609f51d2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:56.732077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28bc9d-6085-45b1-8fb1-6fc0609f51d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:56.732351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28bc9d-6085-45b1-8fb1-6fc0609f51d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:56.733044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28bc9d-6085-45b1-8fb1-6fc0609f51d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:56.733441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28bc9d-6085-45b1-8fb1-6fc0609f51d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:56.734840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28bc9d-6085-45b1-8fb1-6fc0609f51d2 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:56.743985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 239/480] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:18:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:57.646030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99f3b-4407-4601-9710-c375f2c037ca None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:57.646554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99f3b-4407-4601-9710-c375f2c037ca None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:57.646554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99f3b-4407-4601-9710-c375f2c037ca None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:57.648693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 242/481] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:57.653591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbbf2-008a-41a5-92bc-cb0ab7297b1a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:57.653991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbbf2-008a-41a5-92bc-cb0ab7297b1a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:57.654175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbbf2-008a-41a5-92bc-cb0ab7297b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:57.679862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-789bbbf2-008a-41a5-92bc-cb0ab7297b1a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:57.681060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-789bbbf2-008a-41a5-92bc-cb0ab7297b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:57.699657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:57.766617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 240/482] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:57.801732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99f3b-4407-4601-9710-c375f2c037ca None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:57.936532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:57.941227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99f3b-4407-4601-9710-c375f2c037ca None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:57.985396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c50e04-83ca-42f5-930a-449932e22945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:57.986025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c50e04-83ca-42f5-930a-449932e22945 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:57.986305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c50e04-83ca-42f5-930a-449932e22945 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:57.986581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c50e04-83ca-42f5-930a-449932e22945 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.002177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c50e04-83ca-42f5-930a-449932e22945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-51c50e04-83ca-42f5-930a-449932e22945 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:58.006817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:58.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:58.009429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 243/483] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:18:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:58.016394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbbf2-008a-41a5-92bc-cb0ab7297b1a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:58.222294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbbf2-008a-41a5-92bc-cb0ab7297b1a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:58.258920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f588-7995-4f63-94b9-526e45554380 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.259235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f588-7995-4f63-94b9-526e45554380 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.259471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f588-7995-4f63-94b9-526e45554380 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.261106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 244/484] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:58.270752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0834d-97ff-406e-844e-17b34160ff85 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.271274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0834d-97ff-406e-844e-17b34160ff85 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.271558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0834d-97ff-406e-844e-17b34160ff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.273258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:58.274038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.274369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.274718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.275557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:58.275982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:58.277461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:58.287347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 241/485] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:18:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:58.294525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:58.312201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-03a0834d-97ff-406e-844e-17b34160ff85 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:58.313420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-03a0834d-97ff-406e-844e-17b34160ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:58.317609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:58.442373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 245/486] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:18:58.457507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c9d8-4119-4c7b-bf49-2200be431bea None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:58.478975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0834d-97ff-406e-844e-17b34160ff85 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:58.512148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:58.512779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.513054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.513358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.532033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:58.533519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:18:58.555504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:58.556425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:18:58.557747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:18:58.557747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:58.557747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:58.557747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 242/487] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:58.566964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:58.613769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:58.627947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0834d-97ff-406e-844e-17b34160ff85 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:58.660460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:58.672238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48c1d8-d8d4-4259-bd00-2cea8e6f84fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:58.673501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c1d8-d8d4-4259-bd00-2cea8e6f84fd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.674041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c1d8-d8d4-4259-bd00-2cea8e6f84fd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.674537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c1d8-d8d4-4259-bd00-2cea8e6f84fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.689460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48c1d8-d8d4-4259-bd00-2cea8e6f84fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-1c48c1d8-d8d4-4259-bd00-2cea8e6f84fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:58.697580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:58.698447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:18:58.699267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:58.701425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 246/488] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:18:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:58.706487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60b13f-f701-42be-a824-e6ae6d042abc None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.706624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60b13f-f701-42be-a824-e6ae6d042abc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.706863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60b13f-f701-42be-a824-e6ae6d042abc None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.708300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 247/489] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:58.708871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60b13f-f701-42be-a824-e6ae6d042abc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:58.717077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4217dfa-f0f4-48b5-a847-75e47af789d4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:58.753317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663fed2-52bd-4f6a-948d-15da2adf91ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:58.753911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663fed2-52bd-4f6a-948d-15da2adf91ab None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.754192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663fed2-52bd-4f6a-948d-15da2adf91ab None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.754463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663fed2-52bd-4f6a-948d-15da2adf91ab None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.755797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663fed2-52bd-4f6a-948d-15da2adf91ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:58.755797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663fed2-52bd-4f6a-948d-15da2adf91ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:58.756781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663fed2-52bd-4f6a-948d-15da2adf91ab None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:58.774560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 243/490] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:58.774560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e2d9-5c69-4b2f-8129-0feaeeb3b543 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.774560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e2d9-5c69-4b2f-8129-0feaeeb3b543 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.774560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e2d9-5c69-4b2f-8129-0feaeeb3b543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.790819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d3e2e2d9-5c69-4b2f-8129-0feaeeb3b543 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:58.792174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d3e2e2d9-5c69-4b2f-8129-0feaeeb3b543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:58.834018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60b13f-f701-42be-a824-e6ae6d042abc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:58.863592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 244/491] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:58.870086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e2d9-5c69-4b2f-8129-0feaeeb3b543 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:58.874034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:58.874812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:58.875268 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:58.875679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:58.876583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:58.877025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:58.878695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:58.887778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 248/492] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:18:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:58.894879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:58.908482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:58.946879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:58.995596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e2d9-5c69-4b2f-8129-0feaeeb3b543 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.014135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1134362-7e1a-4aa9-8166-f9238d4a9b48 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.030798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.031844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.032435 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.033118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.041840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.043544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.053102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.054568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.055061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.055473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:18:59.058687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:59.059696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:59.060606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 245/493] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:18:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:59.061150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.074936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.076673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:18:59.092435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:59.093582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:18:59.094625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:18:59.094625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:59.094625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:59.101481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 249/494] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:18:59.102503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.171258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.173349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03d5ae-cc06-4509-8996-d148c47f8bf0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.189036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.211867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.212744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.213119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.213518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.219118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddd184-d79f-4142-a1ff-98c449c54fde None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.224751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:18:59.231279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:59.232375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:18:59.233373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:59.240185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 246/495] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:59.241380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.260486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.261526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.261998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.262477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.263393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:59.263852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.265789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.305478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 250/496] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:59.306316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.344077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a0504-70b2-474e-8222-5773701644ba None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.377402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee8c8e-a36f-4610-ae23-18e7b98a26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.377991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee8c8e-a36f-4610-ae23-18e7b98a26ab None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.378267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee8c8e-a36f-4610-ae23-18e7b98a26ab None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.378534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee8c8e-a36f-4610-ae23-18e7b98a26ab None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.379170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee8c8e-a36f-4610-ae23-18e7b98a26ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:59.379481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee8c8e-a36f-4610-ae23-18e7b98a26ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.380864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee8c8e-a36f-4610-ae23-18e7b98a26ab None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.392630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 247/497] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:59.393642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49746a-a8a1-4b2a-b62e-75b455c78781 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.393924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49746a-a8a1-4b2a-b62e-75b455c78781 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.394195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49746a-a8a1-4b2a-b62e-75b455c78781 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.395394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 248/498] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:59.396449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa38e-0f39-4e22-83df-a217aae95a2b None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.396740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa38e-0f39-4e22-83df-a217aae95a2b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.396976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa38e-0f39-4e22-83df-a217aae95a2b None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.398718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 249/499] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:18:59.399410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aa38e-0f39-4e22-83df-a217aae95a2b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.412082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eea1a0-b2c3-4f73-a233-e431a730debf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.432011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.434868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.446250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.446852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.447139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.447405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.448065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:59.448371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.449909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.462893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 251/500] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:59.463535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.505175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aa38e-0f39-4e22-83df-a217aae95a2b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.538957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.539489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.539764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.540036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.560802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d95c2-23f8-4205-afed-8b23f9969298 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.594323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802f37c-8015-4a8a-b74e-3158310402f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.595218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802f37c-8015-4a8a-b74e-3158310402f9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.595495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802f37c-8015-4a8a-b74e-3158310402f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.595763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802f37c-8015-4a8a-b74e-3158310402f9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.603643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802f37c-8015-4a8a-b74e-3158310402f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.605068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802f37c-8015-4a8a-b74e-3158310402f9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.609080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.610593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-6802f37c-8015-4a8a-b74e-3158310402f9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:18:59.617192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:59.618105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:18:59.618995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 252/501] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:59.619556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069c662-3c60-478a-8644-0fda1269fea0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.619804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069c662-3c60-478a-8644-0fda1269fea0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.620263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069c662-3c60-478a-8644-0fda1269fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.637969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9069c662-3c60-478a-8644-0fda1269fea0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:59.639158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9069c662-3c60-478a-8644-0fda1269fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:59.647009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.674219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.706018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 253/502] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:59.707229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c5de-f702-4904-b1ab-789415d20ac4 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.707512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c5de-f702-4904-b1ab-789415d20ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.707784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c5de-f702-4904-b1ab-789415d20ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.724831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f1d3c5de-f702-4904-b1ab-789415d20ac4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:18:59.729932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 250/503] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:18:59.730408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f1d3c5de-f702-4904-b1ab-789415d20ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:18:59.730502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.793098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 254/504] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:18:59.794131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3c5de-f702-4904-b1ab-789415d20ac4 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.826382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43597d-c6c9-4d08-90a1-c55a4f1d618d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.860269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.860900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.861180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.861480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.863423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.875686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.877142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:18:59.892134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:18:59.893220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:18:59.894515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:18:59.894515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:18:59.894515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:18:59.894515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 251/505] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:18:59.894515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.901667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3c5de-f702-4904-b1ab-789415d20ac4 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:18:59.903350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:18:59.936769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be0773-f94f-40ea-a002-af579943815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:18:59.937402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be0773-f94f-40ea-a002-af579943815b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:18:59.937680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be0773-f94f-40ea-a002-af579943815b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:18:59.937948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be0773-f94f-40ea-a002-af579943815b None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:18:59.938664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be0773-f94f-40ea-a002-af579943815b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:18:59.938970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be0773-f94f-40ea-a002-af579943815b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:18:59.940388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be0773-f94f-40ea-a002-af579943815b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:18:59.958781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 255/506] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:18:59.959578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be0773-f94f-40ea-a002-af579943815b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:18:59.998273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31742fb-698c-4994-a403-9cc1e7cb550e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.031035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.031658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.031944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.032218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.042064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:00.047225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:00.048094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:00.048871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 252/507] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:00.049555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.059877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be0773-f94f-40ea-a002-af579943815b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.094005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.094157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.094623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.094929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.095202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.105476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.106933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:00.122015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:00.122937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:00.123772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:00.123772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.123772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:00.123772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 256/508] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:00.124121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.138921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.151011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb26f-39b3-488a-9c4b-23fd381edabd None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.184611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.185242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.185535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.185801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.195884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.197369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:00.211344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:00.212391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:00.213437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:00.213437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.213437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:00.213437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 253/509] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:00.213437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.226332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3ee28-318b-47c3-a1e3-4506694df02f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.261797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.262393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.262670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.262938 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.263572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:00.263871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.265237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.277337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 257/510] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:00.278283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.314279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.318594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29620b-2c13-49c1-ae74-05ae49c6b94c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.350240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.354613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.355221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.355501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.355767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.380377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473b7f-0e2e-4884-9c55-aa9576434a98 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.413969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.414566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.414846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.415118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.415778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:00.416080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.417472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.422758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.424193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.430440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 258/511] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:00.430440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.521160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.525355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26b95-2e7b-422a-a668-7da0ae252688 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.526576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 254/512] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:00.527804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.561399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.562003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.562283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.562555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.563188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:00.563490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.564927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.566322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.578113 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 259/513] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:00.578922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.629409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be77980-db89-4a28-b06e-d0dc544a732d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.662246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.662856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.663670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.663953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.664801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:00.664919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.666283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.677435 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 255/514] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:00.678200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.678635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2ef7-74bb-4147-8913-abd408d6539b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.713231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.713864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.714139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.714409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.722342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.723776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:00.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:00.736265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:00.737307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:00.737844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.738228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 260/515] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:00.738228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.780689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1b5d-bc55-4c1b-a7e8-78066cd8acb1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.790957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.813505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.814130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.814389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.814655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.824855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.826289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.834407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a43e2-6f41-4a28-b8d6-a5985340d4c1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:00.840233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:00.841103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:00.841984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:00.841984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.841984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:00.845576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 256/516] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:00.846368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.869102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.869767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.870043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.870318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.880720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.882216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:00.895841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:00.896717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:00.898153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:00.898153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:00.898153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:00.898153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 261/517] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:00.898372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:00.945042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0040d-7993-462d-80ae-0b10f1f36570 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.954853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:00.979002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:00.979614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:00.979894 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:00.980172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:00.986738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:00.988496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:00.995148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650f892-2993-4fdc-9684-9023a5df8220 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:00.999921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:01.000899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 257/518] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:01.001776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.006200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.030579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.031171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.031464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.031711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.032333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.033812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.054197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 262/519] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:19:00 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:01.055217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.118101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1977e4-2cef-4cbe-8af6-927ecc6f5c92 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.156025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.156745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.157031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.157393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.158133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:01.158442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.159961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.176273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071b9d3-6708-40cb-a650-df1f64676533 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.205208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 258/520] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:01.205208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.220215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.220829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.221107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.221425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.222077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:01.222379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.223766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.233706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.239738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 263/521] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:01.240405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.269224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.313651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313087f4-54a6-4160-97c1-9a81b1eb3b13 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.349489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.349957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce8fb9-a7d4-469b-902a-f80e72d00387 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.350129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.350369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.350667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.351314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:01.351618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.353246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.364418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 259/522] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:01.365218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.395033 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.395558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.395939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.396104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.396791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:01.397102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.398614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.421122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 264/523] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:01.421122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.465170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.478573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c34b2-bcbb-40cb-b640-0923d118d614 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.507490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.519117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.519995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.520271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.520699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.527784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8e6ca-fe13-407c-beab-213cacf0d369 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.566873 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.567485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.567760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.568036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.576396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.577905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:01.589632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:01.590649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 265/524] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:01.591592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.606335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.607864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.695957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.701902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d33511-d0f2-4172-8f31-3248e9c1e46a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.711331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 260/525] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:01.712371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.743393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.744009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.744282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.744548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.751380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.752944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:01.765484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:01.766520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:01.767615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:01.768357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 266/526] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:01.769130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.771847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.834540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485b4c-c8b1-4dfe-a4ce-3dfdbca0b4fb None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.872044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.872225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350d25c-aee9-4ee2-af80-652c273c5b82 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:01.872966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.873535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.873809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.874644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=536ca83c26354a77b1c5d263c57f4f7d) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:01.874947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.876357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.889297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 261/527] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/role_assignments?user.id=536ca83c26354a77b1c5d263c57f4f7d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:01.889297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:01.920082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:01.921048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:01.921445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:01.922188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:01.923297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:01.923673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:01.926150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:01.950828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:01.958293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 267/528] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:01.959786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.013245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c1c5c-2686-4044-9893-b85982626c5b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.032879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:02.050573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.051212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.051485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.051754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.064593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.065773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:02.083848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:02.084785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:02.086268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:02.086268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:02.086268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:02.086268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 262/529] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:02.086492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.094172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331f198-2451-49eb-b020-76bfa8448e83 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.130347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.130965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.131368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.131735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.132994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:02.133344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.134951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.150664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 268/530] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:02.151578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.205541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea054-8838-4454-9777-e55f515f0e59 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.209234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:02.255556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.256852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.258100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.258574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.268065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9126e-27b6-4432-83a6-8c5a4556d2f1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.290061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:02.306310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.306917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.307193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.307464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.308113 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:02.308414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.309862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.327471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 269/531] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:02.328254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.354421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.357207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.436062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b7bc0-5170-4cab-8d3b-20581194bb2f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.480350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 263/532] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:02.481257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.482218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.483080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.483763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.485208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:02.485841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.486857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=9be8c8c5f7f6417e9ab19ef5bce0f060) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:02.487386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.489023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.502980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 270/533] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/role_assignments?user.id=9be8c8c5f7f6417e9ab19ef5bce0f060&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:02.502980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.597684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:02.672780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c8450-5b8c-4e00-a9a1-99c39263bde4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.709454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfed12-6974-41dd-a843-68a4bc9d61fc None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.728695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.729503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.729900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.730483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.738636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.740237 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.747241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.748050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.748496 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.748913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:02.755716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:02.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:02.758370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:02.758784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 264/534] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:02.765318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.769614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.771192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:02.798865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:02.800007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:02.801070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:02.801070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:02.801070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:02.808962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 271/535] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:02.814370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:02.866813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:02.906464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b728d9-e9f9-479a-a37d-593fe5375aa3 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:02.941522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:02.942256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:02.942594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:02.942935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:02.943706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:02.944078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:02.946254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:02.964448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 265/536] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:02.970215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:03.056541 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4073e-8c45-4e0d-87fa-c64f85bb2bac None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:03.061609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:03.075347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fb445-fa92-43e9-a1a6-2f7a46d5400b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:03.105692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:03.113913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591493-f3a1-4378-ad82-119e1cd4cb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:03.114622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591493-f3a1-4378-ad82-119e1cd4cb0a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.114973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591493-f3a1-4378-ad82-119e1cd4cb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.115315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591493-f3a1-4378-ad82-119e1cd4cb0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.116057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db591493-f3a1-4378-ad82-119e1cd4cb0a None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=89d07f733a6c44858ba396c933e370e0) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:03.117222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db591493-f3a1-4378-ad82-119e1cd4cb0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:03.119141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db591493-f3a1-4378-ad82-119e1cd4cb0a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:03.129947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 266/537] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/role_assignments?user.id=89d07f733a6c44858ba396c933e370e0&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:03.174710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:03.175349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.175628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.175901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.176564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:03.176876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:03.178311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:03.187303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a51b67-7d4e-45b1-89c9-6149bffb9542 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.187584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a51b67-7d4e-45b1-89c9-6149bffb9542 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.187858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a51b67-7d4e-45b1-89c9-6149bffb9542 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.190464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 267/538] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:03.195432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369c64-24f1-44ae-97e4-1bfa8c534ed1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.195710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369c64-24f1-44ae-97e4-1bfa8c534ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.195978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369c64-24f1-44ae-97e4-1bfa8c534ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.196535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 272/539] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:03.205063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:03.222455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-af369c64-24f1-44ae-97e4-1bfa8c534ed1 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:03.223677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-af369c64-24f1-44ae-97e4-1bfa8c534ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:03.357220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c71868-7c75-4627-bfae-c7adb671cbdf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:03.364887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 268/540] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:19:03.385088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af369c64-24f1-44ae-97e4-1bfa8c534ed1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:03.418202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:03.418837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.419108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.419373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.430925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:03.432626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:03.446340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:03.447209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:03.448061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:03.452576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 273/541] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:03.453801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:03.476223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:03.567941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d957f-1433-4375-9622-edaa9de3a97e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:03.614525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:03.615149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.615495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.615835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.617362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:03.617758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:03.619535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:03.637703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 274/542] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:03.638900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:03.651161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af369c64-24f1-44ae-97e4-1bfa8c534ed1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:03.701681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:03.733870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:03.734508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.734783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.735053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.745634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:03.754415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:03.759285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:03.760132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:03.761344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 269/543] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:03.768131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:03.769245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54370e8e-286c-4450-86b1-2e41ccb67486 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:03.813845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:03.814512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.814792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.815068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.815854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=8c4588df9de242e8819de0476f75959c) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:03.816166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:03.817685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:03.824551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 275/544] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:03 2022] GET /identity/v3/role_assignments?user.id=8c4588df9de242e8819de0476f75959c&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:03.907190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371499cc-8263-4234-b775-d948c04d84b7 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:03.946473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c560de-585b-4993-bd05-5919f3bed865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:03.947176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c560de-585b-4993-bd05-5919f3bed865 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:03.947461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c560de-585b-4993-bd05-5919f3bed865 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:03.947740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c560de-585b-4993-bd05-5919f3bed865 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:03.948494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c560de-585b-4993-bd05-5919f3bed865 None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:03.948803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c560de-585b-4993-bd05-5919f3bed865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:03.950273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c560de-585b-4993-bd05-5919f3bed865 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:03.960900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 270/545] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:03.967415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:04.089904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c668d90-9169-480e-9bbd-939c02d9fcd4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:04.115128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a507-bdb0-474a-8b13-68ff7094483a None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.115454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a507-bdb0-474a-8b13-68ff7094483a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.115672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a507-bdb0-474a-8b13-68ff7094483a None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.117053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 271/546] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:04.122580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6501029-1bd5-4c3f-90dd-8043f0587afa None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.122865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6501029-1bd5-4c3f-90dd-8043f0587afa None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.123162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6501029-1bd5-4c3f-90dd-8043f0587afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.140328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:04.141437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.141919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.142348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.145680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c6501029-1bd5-4c3f-90dd-8043f0587afa None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:04.146856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c6501029-1bd5-4c3f-90dd-8043f0587afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:04.153238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:04.156272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:04.158141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:04.174828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:04.175807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:04.176647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:04.176647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:04.176647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:04.182845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 276/547] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:04.184560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:04.204957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:04.244935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 272/548] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:04.289373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6501029-1bd5-4c3f-90dd-8043f0587afa None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:04.384084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c9e5-9ebd-4640-9a0b-8c3db814de3a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:04.425750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6501029-1bd5-4c3f-90dd-8043f0587afa None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:04.450571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:04.451563 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.451984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.452396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.453295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:04.453838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:04.454685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:04.455321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.455603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.455874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:04.455991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:04.473333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:04.474412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:04.476628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 273/549] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:04.482703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 277/550] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:04.493004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:04.493781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:04.537678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:04.612156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2e099-0805-4904-897e-e49d8f12e74d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:04.650546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:04.651225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.651503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.651775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.653798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:04.661296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:04.663826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:04.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:04.678011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:04.678987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:04.682107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 274/551] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:04.690062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:04.766309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd41e46-1f2f-47f4-96c6-627fd03ebc99 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:04.830633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:04.831478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.831763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.832031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.832701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:04.832998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:04.834395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:04.849622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:04.850122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 278/552] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:04.857535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:04.873715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008bdf5-4931-4ff0-b0fa-6e8c3768e3fc None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:04.947366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:04.948015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:04.948946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:04.948946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:04.949152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:04.949452 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:04.950891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:04.981438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 275/553] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 10 12:19:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:04.981438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.008198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b21c-7c9d-4ab6-b7e8-925df4fa6ffa None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.023262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.044844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.045804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.046192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.046589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.065403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:05.065403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:05.091548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:05.092575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:05.093475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:05.093475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:05.093475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:05.100495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 279/554] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:05.101944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.120277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c240a-396e-46dd-986c-e35fc96038ed None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.160417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfd0b9-62a8-4360-9b80-cb83df889e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.161346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfd0b9-62a8-4360-9b80-cb83df889e82 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.161631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfd0b9-62a8-4360-9b80-cb83df889e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.161906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfd0b9-62a8-4360-9b80-cb83df889e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.201758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.248137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfdae0-f201-41d2-8843-e1ee24b726cf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.252059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfd0b9-62a8-4360-9b80-cb83df889e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:05.253559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfd0b9-62a8-4360-9b80-cb83df889e82 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:05.286024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c773c5-bb2c-4fcf-b0db-cb4a538fa5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.286652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c773c5-bb2c-4fcf-b0db-cb4a538fa5fd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.286931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c773c5-bb2c-4fcf-b0db-cb4a538fa5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.287323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c773c5-bb2c-4fcf-b0db-cb4a538fa5fd None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.288099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c773c5-bb2c-4fcf-b0db-cb4a538fa5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:05.288911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c773c5-bb2c-4fcf-b0db-cb4a538fa5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:05.290320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c773c5-bb2c-4fcf-b0db-cb4a538fa5fd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:05.303342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 280/555] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:05.304707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41187-9559-46bb-af25-8b47329551dd None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.304994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41187-9559-46bb-af25-8b47329551dd None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.305375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41187-9559-46bb-af25-8b47329551dd None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.306835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 281/556] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:05.311435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41187-9559-46bb-af25-8b47329551dd None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.341314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.375835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 276/557] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:05.377676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dc63-cd33-4206-a4a8-46433b6da61d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.378077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dc63-cd33-4206-a4a8-46433b6da61d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.378359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dc63-cd33-4206-a4a8-46433b6da61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.401256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2d64dc63-cd33-4206-a4a8-46433b6da61d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:05.402792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2d64dc63-cd33-4206-a4a8-46433b6da61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:05.432791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41187-9559-46bb-af25-8b47329551dd None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.449005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.481534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.481534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.481534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.481534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.482128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 277/558] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:05.483253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64dc63-cd33-4206-a4a8-46433b6da61d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.489286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:05.489286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:05.500682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:05.502152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:05.503127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 282/559] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:05.507996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.605472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.617035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64dc63-cd33-4206-a4a8-46433b6da61d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.620826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4715a7e-61a8-4903-bf9c-ed8c47874009 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.655665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.656272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.656556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.656826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.657581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:05.657888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:05.658720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3039292-dae7-49e9-9ae8-f2faf6cc1c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.659334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:05.659431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3039292-dae7-49e9-9ae8-f2faf6cc1c38 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.659620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3039292-dae7-49e9-9ae8-f2faf6cc1c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.659888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3039292-dae7-49e9-9ae8-f2faf6cc1c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.671081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3039292-dae7-49e9-9ae8-f2faf6cc1c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:05.672556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3039292-dae7-49e9-9ae8-f2faf6cc1c38 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:05.677460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 283/560] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:05.682503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-b3039292-dae7-49e9-9ae8-f2faf6cc1c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:05.687575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:05.688526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 278/561] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6c44a-92d6-4b75-8cc8-f5d4f8b1fcab None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6c44a-92d6-4b75-8cc8-f5d4f8b1fcab None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6c44a-92d6-4b75-8cc8-f5d4f8b1fcab None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 279/562] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:05.693255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6c44a-92d6-4b75-8cc8-f5d4f8b1fcab None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.726122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.794424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f558314-ff27-4533-8f0e-bc0bdadf7440 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.802525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6c44a-92d6-4b75-8cc8-f5d4f8b1fcab None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:05.830408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.831425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.831883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.832331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.836136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea6d00-0128-498d-9797-6a608d3dce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:05.836759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea6d00-0128-498d-9797-6a608d3dce63 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.837028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea6d00-0128-498d-9797-6a608d3dce63 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.837313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea6d00-0128-498d-9797-6a608d3dce63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.844387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:05.847640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea6d00-0128-498d-9797-6a608d3dce63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:05.849082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea6d00-0128-498d-9797-6a608d3dce63 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:05.849884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:05.850834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:05.851909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:05.854207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 284/563] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:05.855119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.862956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-8eea6d00-0128-498d-9797-6a608d3dce63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:05.868123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:05.869239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:05.870178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:05.870178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:05.870178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:05.870178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 280/564] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:05.870437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc3b1f-82c4-4e42-a85c-445833aeebad None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:05.870723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc3b1f-82c4-4e42-a85c-445833aeebad None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:05.870993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc3b1f-82c4-4e42-a85c-445833aeebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:05.889233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-80dc3b1f-82c4-4e42-a85c-445833aeebad None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:05.890427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-80dc3b1f-82c4-4e42-a85c-445833aeebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:05.962880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 281/565] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:05.964470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc3b1f-82c4-4e42-a85c-445833aeebad None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:05.986918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:05.988485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7018be3-901c-401d-9a32-7c5f1b9f9a96 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.027918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.028522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.028802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.029074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.029749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:06.030054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.031516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.042937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 285/566] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:06.043853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.089955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc3b1f-82c4-4e42-a85c-445833aeebad None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.109236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.124688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.125314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.125597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.125872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.126542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:06.126851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.128230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.140817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 282/567] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:06.140817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.148870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8367a2c-0fc0-40a5-86d0-f3f9c9b5018c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.181829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.182531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.182813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.183083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.183712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:06.184012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.185398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.197361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 286/568] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:06.198140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.210528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.235530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc866e-066e-4ab4-98cb-3ec8a273713d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.270772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8b5b1-3163-40a4-ae01-7a062a0fec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.271451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8b5b1-3163-40a4-ae01-7a062a0fec3f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.271722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8b5b1-3163-40a4-ae01-7a062a0fec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.271991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8b5b1-3163-40a4-ae01-7a062a0fec3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.282463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8b5b1-3163-40a4-ae01-7a062a0fec3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-cae8b5b1-3163-40a4-ae01-7a062a0fec3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:06.288789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:06.289885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:06.290759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 283/569] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:06.291592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec037f1-c8da-458c-b1d3-2ed57f45ca24 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.291592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec037f1-c8da-458c-b1d3-2ed57f45ca24 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.291592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec037f1-c8da-458c-b1d3-2ed57f45ca24 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.292714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 284/570] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:06.293427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec037f1-c8da-458c-b1d3-2ed57f45ca24 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.299468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267aafec-431b-4686-a58d-60c8c659ad71 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.323378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.334329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.334970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.335244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.335512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.342069 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.343465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:06.355746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:06.356800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:06.357706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 287/571] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:06.358198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.399782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec037f1-c8da-458c-b1d3-2ed57f45ca24 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.432760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0785e12-11cd-4f06-bfc9-c5788e6caa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.433403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0785e12-11cd-4f06-bfc9-c5788e6caa85 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.433675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0785e12-11cd-4f06-bfc9-c5788e6caa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.433942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0785e12-11cd-4f06-bfc9-c5788e6caa85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.435765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.445134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0785e12-11cd-4f06-bfc9-c5788e6caa85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.446604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0785e12-11cd-4f06-bfc9-c5788e6caa85 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.458256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40734e66-c739-45cb-a1ca-21b59f8a1694 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-f0785e12-11cd-4f06-bfc9-c5788e6caa85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:06.460115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:06.461309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:06.462198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:06.462198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:06.462198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:06.462198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 285/572] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:06.462468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf5591-5914-4a4d-8953-e2cc8d35f648 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.462750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf5591-5914-4a4d-8953-e2cc8d35f648 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.463014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf5591-5914-4a4d-8953-e2cc8d35f648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.481299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-59cf5591-5914-4a4d-8953-e2cc8d35f648 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:06.482451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-59cf5591-5914-4a4d-8953-e2cc8d35f648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:06.494331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.494953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.495230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.495502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.502057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.503486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:06.514558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:06.515575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 288/573] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:06.521074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.548208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 286/574] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:06.549034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf5591-5914-4a4d-8953-e2cc8d35f648 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.572935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.629204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba708c-009e-442e-912e-73d35673494e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.648731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf5591-5914-4a4d-8953-e2cc8d35f648 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.662300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.662908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.663791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.664066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.664729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:06.664916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.665041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.666474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.674306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 289/575] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:06.674306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.689539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.690156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.690436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.690708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.691405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:06.691711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.693149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.704389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 287/576] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:06.705162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.774077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e64679-6c66-42bf-90ca-c8b756690afb None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.780884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.804844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431be7d2-5237-4934-848d-2569e7c377de None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.807855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.808452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.808730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.809002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.809672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:06.809975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.811336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.822794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 290/577] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:06.823570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.837855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.838460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.838746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.839016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.839705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:06.840007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:06.841477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:06.853524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 288/578] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:06.854162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.887393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:06.927640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3dac6-fc68-4ae0-be86-e7a642091bcf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.950807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8fea9-f01b-4c11-88e0-c5515ff27a07 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:06.961554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.962206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.962481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.962759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:06.972728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:06.978106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:06.979032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:06.979878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 291/579] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:06.980772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:06.989500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:06.990128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:06.990401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:06.990725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.001488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.002921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.008045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:07.020935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:07.022129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:07.023333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:07.023333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:07.023333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:07.023333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 289/580] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:07.023333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.084476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56c43-1b95-47f6-ab08-0ad66e13a41e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.109323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.117946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.118543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.118820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.119087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.119802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=1fb6a74548184806972a7f8deb30f789) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:07.120106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.121567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.121998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dde421-9bb8-4a4b-9758-073bae0f5af2 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.126809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 292/581] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:06 2022] GET /identity/v3/role_assignments?user.id=1fb6a74548184806972a7f8deb30f789&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:07.127535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.169205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.170005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.170354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.170806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.180246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.181905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:07.194507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:07.195701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 290/582] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:07.197079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.242978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.254630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b339b4-4667-43cb-8bc3-f23c028b43b3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.301451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.302124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.302401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.302673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/536ca83c26354a77b1c5d263c57f4f7d/roles/8c3fb9b334554c2c831b5e08ab522856` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.314268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b4c6d-5c2f-46bb-bd48-9111201d699b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.318996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=536ca83c26354a77b1c5d263c57f4f7d, role_id=8c3fb9b334554c2c831b5e08ab522856)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.320475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.335324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 293/583] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:07 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/536ca83c26354a77b1c5d263c57f4f7d/roles/8c3fb9b334554c2c831b5e08ab522856 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:07.338110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.354615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.354856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.354938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.355199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.355892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:07.356196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.357980 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.372801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 291/584] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:07.373742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.387974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.494372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c85cc-daf6-40f1-92db-eb02a22f08f2 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.499902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c5275-daa0-4085-a70b-d051512095b1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.534780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.536889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.536889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.536889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.537617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:07.539804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.540702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.551908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.552529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.552812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.553085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.553817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:07.554127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.556169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.559184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 292/585] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:07.560205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.572885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.576394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 294/586] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:07.577479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.658004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.694938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ed546-4c07-461b-857a-f2d0e7845fb5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.705191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548308f-6ccf-442b-b5d7-521e481e8a23 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.731631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.732597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.733085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.733585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.743926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.745617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.746644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.747331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.747613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.747888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.759887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.761387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:07.763140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:07.764030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:07.764908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:07.764908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:07.764908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:07.767072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 293/587] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:07.768188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:07.777112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:07.778094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:07.778970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:07.778970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:07.778970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:07.778970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 295/588] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:07.783825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.841367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.887659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c90ce1-a522-4c1e-9bf7-ae742f1daed9 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.909340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:07.910105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc245bb7-552b-4bff-ab6f-5246b2c42645 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:07.931117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.932070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.933096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.933593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.945234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.946909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.947363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:07.948163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:07.948431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:07.948702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:07.949441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:07.949747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:07.951381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:07.964534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:07.965469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:07.967099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:07.967627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 296/589] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:07.967627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:07.974648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 294/590] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:07.974648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.078648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ec69-de73-4242-bcb8-c4f162727cbe None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.093051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:08.113212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528fc18-f8f6-437f-89fa-5393f2aa798f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.115823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.116930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.117884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.117884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.118129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:08.118469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.119861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.132761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 297/591] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:08.137501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.157626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.158542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.159047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.159576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.170849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.172539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:08.187434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:08.188343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:19:08.189251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:08.196020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 295/592] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:08.196020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.245685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:08.259097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef5d8a-c3bf-4933-95ec-53b09516ec22 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.307819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.308426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.308708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.308983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.309743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:08.310052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.311611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c90df-6c0e-4ff7-9503-b0e928d1ba9d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.311722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.334712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 298/593] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:08.335644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.356605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.357572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.358050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.358972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.368200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:08.369978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.371976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:08.387993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:08.388856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:08.389771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:08.393371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 296/594] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:08.394347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.452425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044341c-39ad-4456-a847-23f4a7020ded None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.489242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.489894 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.490177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.490452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.491829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:08.492051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.493572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.518234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 299/595] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:08.518234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.521993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:08.525852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2cec2-67f8-4c33-ba3b-553eb451c8ee None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.566417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.567289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.567778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.568209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.582321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.583838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:08.609297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:08.610224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:08.611091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:08.611091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:08.611091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:08.613641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 297/596] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:08.614841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.641947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:08.656821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e34d79-a1fc-4a75-8a5e-0b95e4d0ff89 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.692779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.693480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.693764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.694042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.695330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:08.695562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.697017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.713821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 300/597] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:08.714631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.749621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a5e5b-0b5a-4ba8-a233-7ec4f9c48e88 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.795043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.795940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.796359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.796780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.811535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.813466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.824608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50649e-c2b9-4b04-9f43-be10aef05436 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:08.832968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:08.834118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:08.835785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:08.835785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:08.835785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:08.842680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 298/598] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:08.842680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.847609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:08.869306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:08.870114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:08.870490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:08.870868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:08.871802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:08.872076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:08.873649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:08.891738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 301/599] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:08.892753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:08.903805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:08.983083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6889adf-6f06-41d2-b790-4068fa4d6107 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.003295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e0fcf-f85a-40e1-8d80-3c71559146c9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.028176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.028988 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.029404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.029796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/9be8c8c5f7f6417e9ab19ef5bce0f060/roles/8c3fb9b334554c2c831b5e08ab522856` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.043926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.044957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.045420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.045705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=9be8c8c5f7f6417e9ab19ef5bce0f060, role_id=8c3fb9b334554c2c831b5e08ab522856)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.046011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.046786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:09.047229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.047663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.049159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.068299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 302/601] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:09.068712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 299/601] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:08 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/9be8c8c5f7f6417e9ab19ef5bce0f060/roles/8c3fb9b334554c2c831b5e08ab522856 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:09.070478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.073038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.157705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:09.189021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:09.215077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a116571-b838-49c5-aeb1-500edf7e34d1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.252417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69337d15-040e-4662-b188-1c063cd00dd6 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.254998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7b58-7f78-431c-81d6-879977cd3d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.255722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7b58-7f78-431c-81d6-879977cd3d74 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.256073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7b58-7f78-431c-81d6-879977cd3d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.256405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7b58-7f78-431c-81d6-879977cd3d74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.264671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7b58-7f78-431c-81d6-879977cd3d74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.266276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7b58-7f78-431c-81d6-879977cd3d74 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-807f7b58-7f78-431c-81d6-879977cd3d74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:09.280432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:09.281451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:09.282335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 303/602] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:09.282811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5f961-8904-4f6a-81c2-b1cb1f9d8c0d None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.283670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5f961-8904-4f6a-81c2-b1cb1f9d8c0d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.283670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5f961-8904-4f6a-81c2-b1cb1f9d8c0d None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.284881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 304/603] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:09.290813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907686f9-91bc-4383-8324-6143dc630603 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.291171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907686f9-91bc-4383-8324-6143dc630603 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.291526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907686f9-91bc-4383-8324-6143dc630603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.303315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.304521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.304805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.305080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.314219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.314375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-907686f9-91bc-4383-8324-6143dc630603 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:09.315679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.315810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-907686f9-91bc-4383-8324-6143dc630603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:09.333289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:09.334200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:09.335893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 300/604] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:09.336374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.397148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 305/605] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:19:09.399116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907686f9-91bc-4383-8324-6143dc630603 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.472361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:09.475979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e183624-bb1c-4c02-a7b5-8282ccde0d73 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.491744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:09.514927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.515454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.515753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.516027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.516752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:09.517059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.518513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.519936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907686f9-91bc-4383-8324-6143dc630603 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.534027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 301/606] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:09.534859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.560802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18f8ab-fb4c-4161-8af8-84e11a987445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.561799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18f8ab-fb4c-4161-8af8-84e11a987445 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.562285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18f8ab-fb4c-4161-8af8-84e11a987445 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.562708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18f8ab-fb4c-4161-8af8-84e11a987445 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.563569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18f8ab-fb4c-4161-8af8-84e11a987445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:09.564053 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18f8ab-fb4c-4161-8af8-84e11a987445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.565669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18f8ab-fb4c-4161-8af8-84e11a987445 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.588575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 306/607] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:09.588575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e57606-8480-44ea-aabb-db4b64892992 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.588575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e57606-8480-44ea-aabb-db4b64892992 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.588575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e57606-8480-44ea-aabb-db4b64892992 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.588575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 307/608] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:09.588575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e57606-8480-44ea-aabb-db4b64892992 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.653309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49b890-8014-4a6c-b001-51bb0efb5172 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.687502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.688137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.688415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.688688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.699477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:09.703085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e57606-8480-44ea-aabb-db4b64892992 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:09.704229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:09.705134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:09.706231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 302/609] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:09.710127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.726306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:09.745678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:09.746313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176159c2-6303-49ad-b89a-50ad28aaa5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.746973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176159c2-6303-49ad-b89a-50ad28aaa5c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.747255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176159c2-6303-49ad-b89a-50ad28aaa5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.747743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176159c2-6303-49ad-b89a-50ad28aaa5c5 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/89d07f733a6c44858ba396c933e370e0/roles/8c3fb9b334554c2c831b5e08ab522856` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.763496 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176159c2-6303-49ad-b89a-50ad28aaa5c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=89d07f733a6c44858ba396c933e370e0, role_id=8c3fb9b334554c2c831b5e08ab522856)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.765024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176159c2-6303-49ad-b89a-50ad28aaa5c5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.779860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 308/610] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:09 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/89d07f733a6c44858ba396c933e370e0/roles/8c3fb9b334554c2c831b5e08ab522856 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:09.781073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f38cc-0ced-4f24-84b1-257ce92b45bc None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.781390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f38cc-0ced-4f24-84b1-257ce92b45bc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.781667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f38cc-0ced-4f24-84b1-257ce92b45bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.803423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-961f38cc-0ced-4f24-84b1-257ce92b45bc None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:09.805253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-961f38cc-0ced-4f24-84b1-257ce92b45bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:09.835770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fba66-5d01-4edd-830f-f5ab1756bc6a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:09.885303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:09.886094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:09.886594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:09.886962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/8c4588df9de242e8819de0476f75959c/roles/8c3fb9b334554c2c831b5e08ab522856` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:09.902144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 309/611] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:09.903740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f38cc-0ced-4f24-84b1-257ce92b45bc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:09.910854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=8c4588df9de242e8819de0476f75959c, role_id=8c3fb9b334554c2c831b5e08ab522856)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:09.915340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:09.939698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 303/612] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:09 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/8c4588df9de242e8819de0476f75959c/roles/8c3fb9b334554c2c831b5e08ab522856 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:09.941062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.045813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:10.060932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:10.064979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f38cc-0ced-4f24-84b1-257ce92b45bc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.065127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abc291-f194-4a9f-b843-537a376f8288 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.103835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.104360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.104482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.104755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.104973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.105072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.105253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.113757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.114602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:10.114923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.117917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.118050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:10.122792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:10.123683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:10.124518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 304/613] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:10.126360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 310/614] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:10.131300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.131644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.240594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c485b-5a29-495a-ad9a-49e9c90ae8e1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.241748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb5887-90c4-442d-b632-f6f6eeb5a07d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.277107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.277918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.278267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.278611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.279419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:10.279533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.279846 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.280249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.280604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.280951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.281570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.292112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 305/615] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:10.293660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.295207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.297759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.310601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:10.311187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:10.312264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:10.313381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:10.313381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:10.313381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:10.314371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 311/616] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:10.320010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.336317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:10.417770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae5c9-6177-4a93-a22e-1e1cc872ac80 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.436869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86476d-df08-413d-b5f6-cfee59348777 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.467437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.468438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.468793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.469570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.474005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.474810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.475159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.476047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.476767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:10.477189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.478980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.483709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.485284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.492587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 312/617] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:10.500759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:10.506207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:10.507143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:10.508479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:10.508479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:10.508479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:10.508836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 306/618] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:10.517976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.592283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:10.670757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f25b-9608-421f-a817-fbfa44524c81 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.683084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:10.718345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbab883-e31b-4bc1-9264-63466afe0ffe None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.754497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.755172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.755455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.755729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.757045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:10.757403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.758478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.758801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.759342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.759725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.760114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.767949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.769537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:10.784321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:10.785637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:10.787194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:10.788277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 307/619] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:10.788386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 313/620] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:10.789826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.792018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:10.902370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6edab5-9d79-458d-9d94-46f8dcd4f546 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:10.934250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:10.940514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:10.941122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:10.941433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:10.941709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:10.942445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:10.942754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:10.944721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:10.964412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 314/621] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:10.969969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:11.063406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc01ff-fa29-4fed-b748-76d42d5babc8 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:11.091896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4b059-582e-4409-86c8-71ffce8428bb None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:11.110329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:11.135485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:11.137228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.137988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.138270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.147859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:11.148541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.148824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.149094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.157463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:11.159477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:11.166367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:11.167842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:11.172278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:11.173213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:11.174887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 308/622] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:11.179589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:11.202731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:11.204937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:11.207028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:11.207028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:11.207028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:11.211680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 315/623] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:11.218070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:11.314944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:11.400679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a9900-4ae9-4cc5-99b2-bba6a9807a0e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:11.418485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23883392-e9c8-4a46-b6e0-f601cced1c88 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:11.438400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7050fa-b5f0-4880-aa02-3f573849be06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:11.439014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7050fa-b5f0-4880-aa02-3f573849be06 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.439291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7050fa-b5f0-4880-aa02-3f573849be06 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.439560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7050fa-b5f0-4880-aa02-3f573849be06 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.440289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7050fa-b5f0-4880-aa02-3f573849be06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:11.440593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7050fa-b5f0-4880-aa02-3f573849be06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:11.442073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7050fa-b5f0-4880-aa02-3f573849be06 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:11.463350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 309/624] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:19:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:11.465293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066da67-36a3-41b0-80c5-c8989a80a9ea None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.465646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066da67-36a3-41b0-80c5-c8989a80a9ea None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.465858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066da67-36a3-41b0-80c5-c8989a80a9ea None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.467627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 310/625] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:11.468948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:11.473345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.473345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.473345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.473345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:11.473345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:11.473345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:11.475094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621470d-2324-4a6b-ad80-93e7f525db15 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.475422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621470d-2324-4a6b-ad80-93e7f525db15 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.475700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621470d-2324-4a6b-ad80-93e7f525db15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.492293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 316/626] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:11.493585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:11.502839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7621470d-2324-4a6b-ad80-93e7f525db15 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:11.504052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7621470d-2324-4a6b-ad80-93e7f525db15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:11.543160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:11.587307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 311/627] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:11.588468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621470d-2324-4a6b-ad80-93e7f525db15 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:11.636330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c815f-3f7d-4a89-bf1d-f48609fe89a0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:11.638407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:11.676218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:11.677293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.677819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.678315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.721560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621470d-2324-4a6b-ad80-93e7f525db15 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:11.759658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:11.760659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.760978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:11.761477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.761956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.762508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:11.769703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:11.771502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:11.784015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:11.784935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:11.785857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:11.788902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 312/628] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:11.790028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:11.819716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:11.896780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 317/629] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:11.896780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:11.898218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:11.909251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb67717-e066-4cc5-8c72-6315dcde1a2a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:11.947277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:11.948339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:11.948928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:11.949455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:11.962689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:11.967974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:11.968902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:11.969757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:11.972431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 313/630] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:11.973537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.028051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab1fa2-3378-41b3-a3ab-4d00749a3a2a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.067834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.071858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.072475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.072755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.073030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.073750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:12.074491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.076257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.093204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4f992-0534-4f99-b58f-00e2108c3b0a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.097224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 318/631] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:12.099023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.131957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.132646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.132863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.133135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.144117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.145593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:12.163607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:12.164779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:12.166032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:12.166032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:12.166032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:12.166032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 314/632] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:12.166286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.182228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.223402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e7d6a-afde-4b2c-bc7a-a027cb017b6a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.261899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a53c33-6ac9-41b2-9747-6a6260a51bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.262505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a53c33-6ac9-41b2-9747-6a6260a51bce None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.262777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a53c33-6ac9-41b2-9747-6a6260a51bce None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.263043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a53c33-6ac9-41b2-9747-6a6260a51bce None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.263696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a53c33-6ac9-41b2-9747-6a6260a51bce None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:12.263995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a53c33-6ac9-41b2-9747-6a6260a51bce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.265383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a53c33-6ac9-41b2-9747-6a6260a51bce None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.275051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5a881-edd1-4dad-9217-62482286a5f3 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.280054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 319/633] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:12.280054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5fbc0-39ac-4dd3-b260-42dee14f01d4 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.280054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5fbc0-39ac-4dd3-b260-42dee14f01d4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.280054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5fbc0-39ac-4dd3-b260-42dee14f01d4 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.280054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 320/634] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:12.280054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5fbc0-39ac-4dd3-b260-42dee14f01d4 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.315034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.315849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.316227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.316596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/1fb6a74548184806972a7f8deb30f789/roles/8c3fb9b334554c2c831b5e08ab522856` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.326673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.334260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=1fb6a74548184806972a7f8deb30f789, role_id=8c3fb9b334554c2c831b5e08ab522856)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.336372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.349305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 315/635] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:12 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/1fb6a74548184806972a7f8deb30f789/roles/8c3fb9b334554c2c831b5e08ab522856 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:12.350207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.414119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5fbc0-39ac-4dd3-b260-42dee14f01d4 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.447929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.462012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7892-2f67-4d8e-a094-9406dc984329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.462631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7892-2f67-4d8e-a094-9406dc984329 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.462907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7892-2f67-4d8e-a094-9406dc984329 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.463180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7892-2f67-4d8e-a094-9406dc984329 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.463834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7892-2f67-4d8e-a094-9406dc984329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:12.464135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7892-2f67-4d8e-a094-9406dc984329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.466137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7892-2f67-4d8e-a094-9406dc984329 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.478769 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ad66-8bd0-4e68-a760-830633d5a2b0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.479450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 321/636] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:12.480785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8056968-0052-4d45-b944-5decb0470e6b None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.481071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8056968-0052-4d45-b944-5decb0470e6b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.481372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8056968-0052-4d45-b944-5decb0470e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.502244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a8056968-0052-4d45-b944-5decb0470e6b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:12.503792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a8056968-0052-4d45-b944-5decb0470e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:12.525242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a42d6a-033a-4581-9e01-7fc2896d5566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.525901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42d6a-033a-4581-9e01-7fc2896d5566 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.526286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42d6a-033a-4581-9e01-7fc2896d5566 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.526473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42d6a-033a-4581-9e01-7fc2896d5566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.536619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a42d6a-033a-4581-9e01-7fc2896d5566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.538142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a42d6a-033a-4581-9e01-7fc2896d5566 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-86a42d6a-033a-4581-9e01-7fc2896d5566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:12.552601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:12.553671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:12.554621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:12.554621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:12.554621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:12.554621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 316/637] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:12.555194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0b19c-b794-4867-a33f-9855aad01e68 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.555194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0b19c-b794-4867-a33f-9855aad01e68 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.555444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0b19c-b794-4867-a33f-9855aad01e68 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.556836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 317/638] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:12.561682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0b19c-b794-4867-a33f-9855aad01e68 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.588882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 322/639] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:12.589780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8056968-0052-4d45-b944-5decb0470e6b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.621771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.689600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0b19c-b794-4867-a33f-9855aad01e68 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.697072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8056968-0052-4d45-b944-5decb0470e6b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.711325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.732163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc86209-af6c-4969-b877-30cf2c0d3e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.732317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.732703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86209-af6c-4969-b877-30cf2c0d3e47 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.732921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.733117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86209-af6c-4969-b877-30cf2c0d3e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.733214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.733408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86209-af6c-4969-b877-30cf2c0d3e47 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.733493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.734042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86209-af6c-4969-b877-30cf2c0d3e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:12.734345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86209-af6c-4969-b877-30cf2c0d3e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.735710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86209-af6c-4969-b877-30cf2c0d3e47 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.741657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:12.743648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:12.751748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 323/640] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:12.751748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1275ea-1fe7-40ee-ae8e-5ddb59995c6d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.751748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1275ea-1fe7-40ee-ae8e-5ddb59995c6d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.751748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1275ea-1fe7-40ee-ae8e-5ddb59995c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:12.759606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:12.760483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:12.761825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:12.765388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 318/641] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:12.766291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.770587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6e1275ea-1fe7-40ee-ae8e-5ddb59995c6d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:12.771772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6e1275ea-1fe7-40ee-ae8e-5ddb59995c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:12.843545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 324/642] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:12.844189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1275ea-1fe7-40ee-ae8e-5ddb59995c6d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.875819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.879272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e174f-63bc-4b8c-8d13-60192d1fdda3 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.919038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.919668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.919944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.920212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:12.930604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:12.935703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:12.936888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:12.938103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:12.941637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 319/643] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:12.942457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:12.952540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1275ea-1fe7-40ee-ae8e-5ddb59995c6d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:12.956056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:12.991144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:12.992159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:12.992511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:12.992847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.000679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.002238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:13.014324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:13.015440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 12:19:13.016740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:13.021611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 325/644] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:13.023004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.056986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b379af-7c0e-4d11-93fe-b8eb9a36f67b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.091890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.092491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.092773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.093041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.093753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:13.094056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.095487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.108717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 320/645] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:13.110153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.137047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:13.142857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6905c6-4310-4d99-9e79-8f798d67e95b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.178099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.179266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.179536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.179809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.190707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:13.196604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:13.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:13.198384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 326/646] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:13.199070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.219255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d08cf-0dde-488d-a533-9cebdf5fd52a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.230498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:13.259946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.260608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.260976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.261368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.262230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:13.262624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.264186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.273716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 321/647] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:13.274677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.315303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17931-2a68-48fc-a163-f62718363ee4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.349102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.349740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.350019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.350832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.351626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:13.351939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.353705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.368309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:13.372363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 327/648] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:13.373488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.388841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5eeb9-4805-4cc6-90f3-d2eec2a81588 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.429352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.429960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.430325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.430565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.431222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=536ca83c26354a77b1c5d263c57f4f7d) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:13.431541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.432966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.439122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 322/649] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/role_assignments?user.id=536ca83c26354a77b1c5d263c57f4f7d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:13.439894 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.479928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:13.500681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94c10c-3d4b-4de8-8d1e-bb88817d0583 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.551492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.552655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.553138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.553574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.554777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:13.555456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.557287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.569645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 328/650] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:13.570960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.587309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb79ae5-824a-4100-8ec2-667cb9d9a118 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.633475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.634546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.634835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.635109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.648740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.650210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.657859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:13.667707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:13.668652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:13.669747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:13.669747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:13.669747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:13.671202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 323/651] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:13.671968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.709456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edd39e-1f8d-4e38-b8a8-aa9aab05123d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.747489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.748098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.748378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.748650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.764386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:13.791635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f368037-c1ef-45ae-b09b-91e29394c5f0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:13.837160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:13.838042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:13.838718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:13.839165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:13.839908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.842502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.852459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:13.854893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:13.874464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:13.875626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:13.876810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:13.876810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:13.876810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:13.879081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 324/652] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:13.884702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:13.958273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:13.982407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 329/653] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:13.983594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.017278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca792feb-bb71-4b58-9041-02ba22cf5764 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.043775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:14.065208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.066471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.066955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.068015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.069729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:14.070067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.071467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.088790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 325/654] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:14.089569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.113150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288fc89-71e1-4712-8b9e-5ba962eae3d1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.161332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.162864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.164145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.164572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.165395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:14.165841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.167371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.188228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 330/655] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:14.188228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.200858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ee644-9bc3-46b0-b571-1fab9c69f45e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.212621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:14.240226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.241640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.242057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.242470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.253457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.255032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:14.272748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:14.273918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:14.275097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:14.275097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:14.275097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:14.280782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 326/656] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:14.281902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.326877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64da578-eb1b-4561-9a8c-992207a2a2a1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.330642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:14.369202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.369991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.370393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.370854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.378846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.380659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:14.392964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:14.393944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 331/657] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:14.394871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.403594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e491850-fe56-4f69-b683-484e65fe4f88 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.439446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9495f-0c6c-4550-8fd1-ec95168994e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.440551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9495f-0c6c-4550-8fd1-ec95168994e6 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.440827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9495f-0c6c-4550-8fd1-ec95168994e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.441090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9495f-0c6c-4550-8fd1-ec95168994e6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.449892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9495f-0c6c-4550-8fd1-ec95168994e6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.451622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9495f-0c6c-4550-8fd1-ec95168994e6 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-44d9495f-0c6c-4550-8fd1-ec95168994e6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:14.463376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:14.464307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 327/658] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fc562-4ffe-49b3-b4bb-93eaf03f50e6 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.470170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fc562-4ffe-49b3-b4bb-93eaf03f50e6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.470890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fc562-4ffe-49b3-b4bb-93eaf03f50e6 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.470890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 328/659] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:14.470890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fc562-4ffe-49b3-b4bb-93eaf03f50e6 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.496285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:14.509542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a853676-867e-4097-845c-4e793b83dae8 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.544045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.544763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.545038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.545343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.545985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:14.546289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.547657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.564491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 332/660] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:14.564491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.586593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:14.586907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fc562-4ffe-49b3-b4bb-93eaf03f50e6 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.624856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e1fbf-ae87-4a60-a840-baca0ec6bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.625450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e1fbf-ae87-4a60-a840-baca0ec6bb0d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.625704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e1fbf-ae87-4a60-a840-baca0ec6bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.625969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e1fbf-ae87-4a60-a840-baca0ec6bb0d None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.626691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e1fbf-ae87-4a60-a840-baca0ec6bb0d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:14.626990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e1fbf-ae87-4a60-a840-baca0ec6bb0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.628417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e1fbf-ae87-4a60-a840-baca0ec6bb0d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.650551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 329/661] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:14.650551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5f572-de1e-4ccd-b483-00360ec506d7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.650551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5f572-de1e-4ccd-b483-00360ec506d7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.650551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5f572-de1e-4ccd-b483-00360ec506d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.668577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5ea5f572-de1e-4ccd-b483-00360ec506d7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:14.669791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5ea5f572-de1e-4ccd-b483-00360ec506d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:14.673789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78784ec-7bfc-4baf-9634-d28aadd46a90 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.710727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.711335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.711612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.711887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.712607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:14.712910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.714369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.736832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 333/662] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:14.736832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.750612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 330/663] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:14.751694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5f572-de1e-4ccd-b483-00360ec506d7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.773940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:14.817530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:14.836612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced06cc-2c40-45b0-943b-daf3b94a6b28 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.872344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.873051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.873354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.873633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.875649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5f572-de1e-4ccd-b483-00360ec506d7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:14.880025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.881510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:14.892224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:14.893310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:14.894261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 334/664] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:14.894744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:14.917706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:14.918597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:14.919026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:14.919436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:14.988848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:14.990392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.003807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.102094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 331/665] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:15.103213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.122288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae22ff4-fbda-4eb1-af55-81818c10e0f2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.156658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.157346 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.157720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.158090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.170405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.216615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eb08d-c36f-4b91-87e6-5f24e404feb8 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.228987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.229157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.231029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.256422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.258014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.258475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.259005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.270845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:15.276575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:15.277545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:15.278393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:15.285472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 332/666] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:15.286697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.333519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 335/667] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:15.334360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.384669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.397078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0d5b-16f6-4c0d-a524-3615e9cad5f8 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.433533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.434139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.434413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.434682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.435363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:15.435666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.437089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.446754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed4865-89c2-4cbf-b884-4996a6235a8a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.448690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 333/668] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:15.449345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.482605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.483543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.484010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.484345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.485721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.499129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.500578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:15.514739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:15.515669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:15.516564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:15.516564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:15.516564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:15.516564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 336/669] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:15.516564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.566695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49145cd2-7ea2-47c8-a196-1288ee5c5f73 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.603782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.604398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.604680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.604983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.605682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:15.605988 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.607365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.615675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 334/670] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:15.617109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.623593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.631700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90988d-1e63-4408-adc6-a9c9cbb22518 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.665136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.665784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.666058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.666329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.676443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.677901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:15.692706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:15.693789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:15.695063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:15.695063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:15.695063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:15.695063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 337/671] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:15.695063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.708287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.719032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60b91-f0ff-48b4-80d3-d2358806ee89 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.752877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1d8a0-ce92-4b42-9d43-7cf794cbfd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.753509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1d8a0-ce92-4b42-9d43-7cf794cbfd46 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.753785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1d8a0-ce92-4b42-9d43-7cf794cbfd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.754052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1d8a0-ce92-4b42-9d43-7cf794cbfd46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.754761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1d8a0-ce92-4b42-9d43-7cf794cbfd46 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=9be8c8c5f7f6417e9ab19ef5bce0f060) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:15.755064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1d8a0-ce92-4b42-9d43-7cf794cbfd46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.756522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1d8a0-ce92-4b42-9d43-7cf794cbfd46 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.762103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 335/672] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/role_assignments?user.id=9be8c8c5f7f6417e9ab19ef5bce0f060&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:15.763250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075c931-44e4-409d-8dc0-82eca364ce03 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.763535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075c931-44e4-409d-8dc0-82eca364ce03 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.763813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075c931-44e4-409d-8dc0-82eca364ce03 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.769329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 336/673] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:15.770646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075c931-44e4-409d-8dc0-82eca364ce03 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.800902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8a2b6-173f-474c-93a7-92f39b437903 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.839270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.839951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.840167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.840440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.841100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:15.841540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.843049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.862616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 338/674] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:15.862616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:15.870469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:15.889388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075c931-44e4-409d-8dc0-82eca364ce03 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.932285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb708e8-295a-4245-9d32-818fc11871b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:15.932924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb708e8-295a-4245-9d32-818fc11871b8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.933210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb708e8-295a-4245-9d32-818fc11871b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.933514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb708e8-295a-4245-9d32-818fc11871b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.944665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb708e8-295a-4245-9d32-818fc11871b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:15.945909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb708e8-295a-4245-9d32-818fc11871b8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-5eb708e8-295a-4245-9d32-818fc11871b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:15.962719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:15.963822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:15.964724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:15.964724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:15.964724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:15.964724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 337/675] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:15.965443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c5685-66e8-4c2c-9ad6-dcc3853f1267 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:15.965724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c5685-66e8-4c2c-9ad6-dcc3853f1267 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:15.965997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c5685-66e8-4c2c-9ad6-dcc3853f1267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:15.984384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-161c5685-66e8-4c2c-9ad6-dcc3853f1267 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:15.985628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-161c5685-66e8-4c2c-9ad6-dcc3853f1267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:15.987497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ce3b-5493-4390-9f38-8b059d91f1c6 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:15.987872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.026949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.027558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.027837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.028773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.028773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:16.029046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.030439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.042589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 339/676] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:16.043231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.066116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 338/677] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:16.072558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c5685-66e8-4c2c-9ad6-dcc3853f1267 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.113092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.151525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10097a-4e03-4b5f-823f-1c8ceba0f48b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.186556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.187210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.187439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.187712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.188216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c5685-66e8-4c2c-9ad6-dcc3853f1267 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.195903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.197722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:16.209584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:16.210969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:16.211854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 340/678] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:16.212363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.227908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.228817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.229097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.229539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.230172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:16.230470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.231934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.238498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.244673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 339/679] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:16.245550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.328660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51b366-de2f-4fdb-90ab-93510218b6da None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.355020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.355183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6c0dc-ffa6-4ea9-a101-3d1e2ebe8127 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.364453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.365082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.365391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.365667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.372976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.374516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:16.385876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:16.386738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:16.387609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 341/680] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:16.388110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.395983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.396617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.397664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.397787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.408346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:16.414018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:16.414982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 340/681] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:16.415819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.484176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.497115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65eaa5-cdc6-4c86-9c0b-d50dfde268b2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.527934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16947-6b57-444b-9dce-01d68b12c26c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.532402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.533007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.533306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.533583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.534272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:16.534707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.536141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.554874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 342/682] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:16.555745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.561881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.562507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.562782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.563049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.571155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.575453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:16.587856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:16.588816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:16.589823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 341/683] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:16.590418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.619927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.674972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034c6e3-7fa0-4bcc-a3c9-b89ac7c2d8c5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.697809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c93eb2-0dee-4e0d-ba2b-43c49cae5639 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.707835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.708418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.708691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.708953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.710182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:16.710489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.711906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.715721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.723844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 343/684] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:16.724558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.741021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.741589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.741863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.742184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.742837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:16.743136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.744493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.757644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 342/685] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:16.757644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.830331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2ccb-155e-4157-af8d-5026e9bd9c89 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.839196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:16.866261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.866857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.867137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.867406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.868125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=89d07f733a6c44858ba396c933e370e0) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:16.868433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.868531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a027d-60df-41db-9eed-73e9c1c665e9 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:16.869905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.875809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 344/686] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/role_assignments?user.id=89d07f733a6c44858ba396c933e370e0&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:16.876801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.918718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:16.919360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:16.919638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:16.919909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:16.920725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:16.921035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:16.922606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:16.942951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 343/687] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:16.943680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:16.994321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:17.016198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc78ab-0257-48ed-9755-26f3d210422e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.064879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.065524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.065807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.066080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.066811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=8c4588df9de242e8819de0476f75959c) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:17.067116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.068629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.076298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 345/688] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/role_assignments?user.id=8c4588df9de242e8819de0476f75959c&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:17.076988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.079816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a7e67-edc8-4dce-b13d-9ff1f070163b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.121922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.122584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.122861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.123132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.140607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.142357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.151164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:17.164243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:17.165319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:17.166206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:17.166206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:17.166206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:17.166206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 344/689] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:17.171427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.244943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffc49-9048-4612-9b92-934c453357d4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.281121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:17.283483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.284377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.285071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.285559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.292282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c88c5e-f360-438a-9dea-29af96227c08 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.328387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.329001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.329294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.329575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.330222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:17.330523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.332047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.345334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 345/690] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:17.350774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.363442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.365224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.401099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:17.463599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0090cfd-b371-422c-ad9d-5397f6edeb07 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.472990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 346/691] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:17.482331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.512750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.513933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.514561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.515034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.525129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.527794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:17.543964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:17.544987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:17.545960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:17.547228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 346/692] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:17.554316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.569544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:17.627382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806c2dd-6d2d-4acc-9927-669974f98651 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.663173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.664157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.664637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.665166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.667379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:17.674432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011868f0-185c-4327-8378-0eff12d833dc None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.678913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.680602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:17.696038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:17.697008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:17.697952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:17.697952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:17.697952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:17.700033 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 347/693] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:17.710235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.715139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.715753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.716272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.716272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.717068 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:17.717412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.718900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.739709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 347/694] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:17.749317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.829173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01100a80-4f30-4611-9088-8c956dd30bbf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.864783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.865777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.866692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.867094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.867876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:17.868339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:17.869879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:17.876352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:17.883313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 348/695] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:17.901076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:17.921222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156d574-f3eb-4760-95ef-4abc3fe00cfe None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:17.960114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f512c6-71be-45a0-9e38-354e032a4bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:17.960727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f512c6-71be-45a0-9e38-354e032a4bea None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:17.961001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f512c6-71be-45a0-9e38-354e032a4bea None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:17.961284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f512c6-71be-45a0-9e38-354e032a4bea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:17.977979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:18.038628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f512c6-71be-45a0-9e38-354e032a4bea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.040990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f512c6-71be-45a0-9e38-354e032a4bea None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.047823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f6a15-744f-4f4c-8e9f-3d1a5032f9f3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.087197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d5b33-ce77-487a-b2f5-37b74bd2f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.087519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5b33-ce77-487a-b2f5-37b74bd2f642 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.087985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5b33-ce77-487a-b2f5-37b74bd2f642 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.088391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5b33-ce77-487a-b2f5-37b74bd2f642 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.097864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d5b33-ce77-487a-b2f5-37b74bd2f642 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.099486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d5b33-ce77-487a-b2f5-37b74bd2f642 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-676d5b33-ce77-487a-b2f5-37b74bd2f642 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:18.112343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:18.113441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:18.114325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:18.119211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 349/696] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:18.122514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60255689-2838-4656-9423-74d936780df5 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.122514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60255689-2838-4656-9423-74d936780df5 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.122913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60255689-2838-4656-9423-74d936780df5 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.125019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 350/697] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:18.129760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60255689-2838-4656-9423-74d936780df5 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.135885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:18.182421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 348/698] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:18.184739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36f7e-3ef7-4d05-8a2e-e4f63ea1d44b None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.185526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36f7e-3ef7-4d05-8a2e-e4f63ea1d44b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.186032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36f7e-3ef7-4d05-8a2e-e4f63ea1d44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.206543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b9a36f7e-3ef7-4d05-8a2e-e4f63ea1d44b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:18.207736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b9a36f7e-3ef7-4d05-8a2e-e4f63ea1d44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:18.255117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60255689-2838-4656-9423-74d936780df5 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.266056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:18.285288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 349/699] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:18.285288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36f7e-3ef7-4d05-8a2e-e4f63ea1d44b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.314743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.316129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.316870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.317396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.318458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:18.319816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.321931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.337797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 351/700] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:18.338664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.401307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:18.406361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a36f7e-3ef7-4d05-8a2e-e4f63ea1d44b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.447142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.447835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.448114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.448384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.463313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.465317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.471865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811e6c9-eaa8-496c-88dd-76e3a16421c2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:18.485084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:18.486162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:18.487201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:18.487201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:18.487201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:18.487201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 350/701] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:18.491765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.508941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.509624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.509900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.510171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.521132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.523215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:18.537670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:18.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:18.541904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:18.541904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:18.541904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:18.541904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 352/702] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:18.541904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.559576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:18.606332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bc498-b7dd-45e6-8c5d-dcd9e6fa631b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.644184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.645361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.645794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.646251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.647190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=1fb6a74548184806972a7f8deb30f789) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:18.647679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.649394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.655802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 351/703] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/role_assignments?user.id=1fb6a74548184806972a7f8deb30f789&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:18.656923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.668694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e6012-da5f-4d22-8646-d975dedcfa99 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.674554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:18.718583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.719478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.719981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.720418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.721365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:18.721876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.723625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.755233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 353/704] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:18.761404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.776778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4cbd4e-658d-41f8-ace8-cc744459e3a1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.850557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.851089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.851367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.851635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.852280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:18.852822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.853616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:18.854397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.873656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 352/705] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:18.879051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bd72b-742f-4c0d-ab0b-2c76adb88bbf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:18.880210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.920309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:18.921307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:18.921849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:18.922125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:18.929919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:18.932052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:18.944794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:18.946009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:18.946941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 354/706] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:18.951983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:18.972608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:19.007607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414aea-74f4-4272-a1de-a58a16cfa36c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:19.058349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:19.059540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.060330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.060819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.062323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:19.063985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:19.078531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074769c-ace0-46a5-83cb-1509928d23d0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:19.093680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 353/707] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:19:18 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:19.120048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81716a12-39ee-4ba2-98e6-2d1cc0245438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:19.120587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716a12-39ee-4ba2-98e6-2d1cc0245438 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.120867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716a12-39ee-4ba2-98e6-2d1cc0245438 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.121140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716a12-39ee-4ba2-98e6-2d1cc0245438 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.121883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81716a12-39ee-4ba2-98e6-2d1cc0245438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:19.122189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81716a12-39ee-4ba2-98e6-2d1cc0245438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:19.123656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81716a12-39ee-4ba2-98e6-2d1cc0245438 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:19.133110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:19.139182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 355/708] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:19.145970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:19.260353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e736a6c-49b8-494c-bb86-b1fd005f911b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:19.293808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:19.298455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe5b5f-6e13-4f5f-b3da-f623bd2cda7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:19.299395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5b5f-6e13-4f5f-b3da-f623bd2cda7a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.299954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5b5f-6e13-4f5f-b3da-f623bd2cda7a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.300525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5b5f-6e13-4f5f-b3da-f623bd2cda7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.301500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe5b5f-6e13-4f5f-b3da-f623bd2cda7a None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=536ca83c26354a77b1c5d263c57f4f7d) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:19.302024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe5b5f-6e13-4f5f-b3da-f623bd2cda7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:19.304334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe5b5f-6e13-4f5f-b3da-f623bd2cda7a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:19.310629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 354/709] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:19 2022] GET /identity/v3/role_assignments?user.id=536ca83c26354a77b1c5d263c57f4f7d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:19.579122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2036d6-dbdc-47a1-8518-8736e2273629 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.579651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2036d6-dbdc-47a1-8518-8736e2273629 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.579651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2036d6-dbdc-47a1-8518-8736e2273629 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.581184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 356/710] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:19.587443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592ee02-9570-4000-a9bf-fe45b7810640 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.587972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592ee02-9570-4000-a9bf-fe45b7810640 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.588537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592ee02-9570-4000-a9bf-fe45b7810640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.605987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09528ab-ac1a-43f9-a068-dc1af890e6ea None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.606274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09528ab-ac1a-43f9-a068-dc1af890e6ea None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.606652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09528ab-ac1a-43f9-a068-dc1af890e6ea None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.608533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 357/711] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:19.611051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8592ee02-9570-4000-a9bf-fe45b7810640 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:19.612266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8592ee02-9570-4000-a9bf-fe45b7810640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:19.615235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b7b1d-7f72-4b6d-aa97-6c746b00cb9d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.615781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b7b1d-7f72-4b6d-aa97-6c746b00cb9d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.616270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b7b1d-7f72-4b6d-aa97-6c746b00cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.638749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-367b7b1d-7f72-4b6d-aa97-6c746b00cb9d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:19.639949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-367b7b1d-7f72-4b6d-aa97-6c746b00cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:19.710021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 355/712] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:19:19.753248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 358/713] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:19.773108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592ee02-9570-4000-a9bf-fe45b7810640 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:19.784033 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b7b1d-7f72-4b6d-aa97-6c746b00cb9d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:19.924199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592ee02-9570-4000-a9bf-fe45b7810640 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:19.949155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:19.966286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b7b1d-7f72-4b6d-aa97-6c746b00cb9d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:19.967503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:19.968239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:19.968518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:19.968788 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] PATH_INFO: `/v3/services/network` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:19.969460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:19.970894 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:19.988310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:19.989446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:19.990304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:19.995552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 356/714] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:19 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:19.996606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:20.002419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:20.006827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:20.007503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:20.007787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:20.008065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:20.008789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:20.010231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:20.021507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:20.023063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:20.024092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 359/715] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 10 12:19:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:20.034287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:20.123131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485d876-7531-48cf-a988-0e84850d401d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:20.168858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:20.169498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:20.169775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:20.170047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:20.170779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] RBAC: Adding query filter params (name=network) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:20.171141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:20.172641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:20.185008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 357/716] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:20.190802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:20.198010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b8d84-95e8-4e42-bfda-391a6e91ee83 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:20.250712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:20.251345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:20.251625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:20.251893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:20.252632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:20.252941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:20.254348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:20.260006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 360/717] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 10 12:19:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:20.265760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:20.272985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:20.329599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7c128-8915-4037-b9c5-1db70b110d38 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:20.364909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1129c-4176-482d-b74a-3d4db9cf5a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:20.365642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1129c-4176-482d-b74a-3d4db9cf5a9c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:20.366515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1129c-4176-482d-b74a-3d4db9cf5a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:20.366795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1129c-4176-482d-b74a-3d4db9cf5a9c None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:20.367548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1129c-4176-482d-b74a-3d4db9cf5a9c None admin] RBAC: Adding query filter params (type=network) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:20.367909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1129c-4176-482d-b74a-3d4db9cf5a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:20.369511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1129c-4176-482d-b74a-3d4db9cf5a9c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:20.370138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:20.376180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 358/718] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:20.448442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ee43-6bf3-4465-aba9-1388a92ff1ab None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:20.499714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878498e-441a-4303-ac44-eadd4b080c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:20.500339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878498e-441a-4303-ac44-eadd4b080c86 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:20.500618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878498e-441a-4303-ac44-eadd4b080c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:20.500890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878498e-441a-4303-ac44-eadd4b080c86 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:20.501555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878498e-441a-4303-ac44-eadd4b080c86 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:20.501863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878498e-441a-4303-ac44-eadd4b080c86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:20.503278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878498e-441a-4303-ac44-eadd4b080c86 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:20.510464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 361/719] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 10 12:19:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:21.222617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97a022-b83c-4220-baa6-7c7f102a1483 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.223162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97a022-b83c-4220-baa6-7c7f102a1483 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.223162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97a022-b83c-4220-baa6-7c7f102a1483 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.224753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 359/720] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:21.238795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3f9a1-1300-497c-ab11-a871b1418c9a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.239289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3f9a1-1300-497c-ab11-a871b1418c9a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.239366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3f9a1-1300-497c-ab11-a871b1418c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.262338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f5f3f9a1-1300-497c-ab11-a871b1418c9a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:21.263987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f5f3f9a1-1300-497c-ab11-a871b1418c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:21.366642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 362/721] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:21.400091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97a022-b83c-4220-baa6-7c7f102a1483 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:21.633708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97a022-b83c-4220-baa6-7c7f102a1483 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:21.651235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:21.669248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15038c14-ea14-463a-bd25-b469852e977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:21.669926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15038c14-ea14-463a-bd25-b469852e977e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.670446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15038c14-ea14-463a-bd25-b469852e977e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.670991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15038c14-ea14-463a-bd25-b469852e977e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.681657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15038c14-ea14-463a-bd25-b469852e977e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-15038c14-ea14-463a-bd25-b469852e977e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:21.686304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:21.687230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:21.688084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 360/722] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:21.698299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3f9a1-1300-497c-ab11-a871b1418c9a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:21.783698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:21.824641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb110ff-d005-4d1c-80a0-6c5b1dea6b64 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.824906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb110ff-d005-4d1c-80a0-6c5b1dea6b64 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.825167 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb110ff-d005-4d1c-80a0-6c5b1dea6b64 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.826783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 361/723] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:21.831762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59589b-0678-43d6-8bf3-b0667a1d7100 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.832042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59589b-0678-43d6-8bf3-b0667a1d7100 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.832325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59589b-0678-43d6-8bf3-b0667a1d7100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.843825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3f9a1-1300-497c-ab11-a871b1418c9a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:21.851423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ad59589b-0678-43d6-8bf3-b0667a1d7100 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:21.852931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ad59589b-0678-43d6-8bf3-b0667a1d7100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:21.883153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be9035-1e97-48ac-b399-6512c17162c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:21.883910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9035-1e97-48ac-b399-6512c17162c6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.884196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9035-1e97-48ac-b399-6512c17162c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.884468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9035-1e97-48ac-b399-6512c17162c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.885140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be9035-1e97-48ac-b399-6512c17162c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:21.885476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be9035-1e97-48ac-b399-6512c17162c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:21.886866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be9035-1e97-48ac-b399-6512c17162c6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:21.895406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 363/724] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:21.896587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40b664-d03d-4080-90ce-379a225d9d4e None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.897457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40b664-d03d-4080-90ce-379a225d9d4e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.897698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40b664-d03d-4080-90ce-379a225d9d4e None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.899174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 364/725] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:21.905834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40b664-d03d-4080-90ce-379a225d9d4e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:21.950767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 362/726] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 12:19:21.958380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56cc65-6896-4279-8b30-418f6649d500 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:21.958993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56cc65-6896-4279-8b30-418f6649d500 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:21.960024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56cc65-6896-4279-8b30-418f6649d500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:21.994981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fc56cc65-6896-4279-8b30-418f6649d500 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:21.996499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fc56cc65-6896-4279-8b30-418f6649d500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:22.016162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40b664-d03d-4080-90ce-379a225d9d4e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.031024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.055479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.056425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.056908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.057414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.067961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.070234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:22.085783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:22.086722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:22.087610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:22.087610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:22.087610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:22.089408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 363/727] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:22.089821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 365/728] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:22.090493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56cc65-6896-4279-8b30-418f6649d500 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.101308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.128581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.205965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56cc65-6896-4279-8b30-418f6649d500 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.227277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc08139-4b2b-4997-86cf-fb6689702147 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.244121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.245370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.245820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.246255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.257706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:22.262412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:22.263321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:22.264158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:22.266295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 364/729] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:22.267428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.271701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.272915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.273455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.274834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.275644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:22.276098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.277682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.299101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 366/730] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:22.299101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.326467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.390238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.399146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5f30c-a13d-43d6-ba13-8d742d75b044 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.424559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d40c3-1c9b-4d7e-8493-ec4e087d7149 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.440939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.441640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.441918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.442192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.455313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:22.460161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:22.461088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:22.461984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 365/731] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:22.462672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.470625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.471512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.471992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.472476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.473371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:22.473893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.475509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.484652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 367/732] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:22.485703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.569753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818604e-acc6-40ad-ae3b-a3ff50842217 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.579944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.601580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da325b5b-b748-43d3-9c0f-3f904e0b5c30 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.607634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.608298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.608583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.608853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.618072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.619615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:22.632364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:22.633362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:22.634354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:22.638107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 366/733] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:22.639041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.644483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.645368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.645947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.646434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.647341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:22.647846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.649576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.657871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.663614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 368/734] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:22.664800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.754304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80634f8c-5bf1-4c6c-9318-69e887f6a9a5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.774424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87724e86-4938-4fd1-94fb-2d0a65e98dec None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.789658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.790285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.790565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.790835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.801844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.803876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.814290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4435-1846-425e-8a16-bad227dba7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.815114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4435-1846-425e-8a16-bad227dba7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.815592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4435-1846-425e-8a16-bad227dba7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.816055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4435-1846-425e-8a16-bad227dba7d0 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.817015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4435-1846-425e-8a16-bad227dba7d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:22.817540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4435-1846-425e-8a16-bad227dba7d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:22.817646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:22.818523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:22.819482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:22.819482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:22.819482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:22.819482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 367/735] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:22.819482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.821602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4435-1846-425e-8a16-bad227dba7d0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.843502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 369/736] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:22.845317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5205-42f7-486d-ab63-84051729e431 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.845671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5205-42f7-486d-ab63-84051729e431 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.846008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5205-42f7-486d-ab63-84051729e431 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.849393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 370/737] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:22.852095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5205-42f7-486d-ab63-84051729e431 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:22.861748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.897157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:22.934757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf3bc9-9c3f-4c2f-8d91-137638a08428 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.963489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5205-42f7-486d-ab63-84051729e431 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:22.967346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:22.968092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:22.968469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:22.968837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:22.979909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:22.981489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:22.996598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:23.002218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:23.003339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:23.003339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:23.003339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:23.003339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 368/738] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:23.003339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.004917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27fbd8-cc32-474c-8623-2340718f4b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.005716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27fbd8-cc32-474c-8623-2340718f4b11 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.006000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27fbd8-cc32-474c-8623-2340718f4b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.006308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27fbd8-cc32-474c-8623-2340718f4b11 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.007801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27fbd8-cc32-474c-8623-2340718f4b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:23.008107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27fbd8-cc32-474c-8623-2340718f4b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.009604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27fbd8-cc32-474c-8623-2340718f4b11 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.021955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 371/739] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:23.023292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87826-6755-4796-8316-ec8fc9093346 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.023674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87826-6755-4796-8316-ec8fc9093346 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.024049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87826-6755-4796-8316-ec8fc9093346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.042352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-98b87826-6755-4796-8316-ec8fc9093346 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:23.043660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-98b87826-6755-4796-8316-ec8fc9093346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:23.107181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.112676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf9c8-e552-4abd-a88d-1441f1745d74 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.125589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 372/740] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:23.126935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b87826-6755-4796-8316-ec8fc9093346 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.134502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.150086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.150758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.151133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.151502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.218481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.219911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.230625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b87826-6755-4796-8316-ec8fc9093346 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.263772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.264363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.264640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.264906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.265572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:23.265874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.279916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 373/741] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:23.280720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.318318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 369/742] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 12:19:23.319150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.336324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.341423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.379483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b113f-0a27-41db-b27f-905eef1bd31b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.412141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d08c1d-10d1-4038-b954-f300e63ad01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.413373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d08c1d-10d1-4038-b954-f300e63ad01a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.413653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d08c1d-10d1-4038-b954-f300e63ad01a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.413919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d08c1d-10d1-4038-b954-f300e63ad01a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.417727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1a626-5903-487a-81c3-4c42feafeffb None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.421065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d08c1d-10d1-4038-b954-f300e63ad01a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.422528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d08c1d-10d1-4038-b954-f300e63ad01a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-12d08c1d-10d1-4038-b954-f300e63ad01a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:23.433259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:23.434407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 374/743] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:19:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:23.435531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83840-c983-458f-a39c-0a1fc31d3449 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.436307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83840-c983-458f-a39c-0a1fc31d3449 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.436307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83840-c983-458f-a39c-0a1fc31d3449 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.437125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 375/744] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:23.437830 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83840-c983-458f-a39c-0a1fc31d3449 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.457660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.458261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.458538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.458808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.459944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.461353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.478633 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 370/745] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:23 2022] POST /identity/v3/services => generated 240 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:23.479432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.570654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83840-c983-458f-a39c-0a1fc31d3449 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.581078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.590514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.608911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64dd3f-6266-49c8-954f-2ff390f6f621 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.617331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.618076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.618430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.618776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.626422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.628309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:23.642489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:23.643501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 376/746] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:19:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:23.644561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.653198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7f159-46c0-4690-b262-fb0baf3471a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.653871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f159-46c0-4690-b262-fb0baf3471a0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.654151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f159-46c0-4690-b262-fb0baf3471a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.654422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f159-46c0-4690-b262-fb0baf3471a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.667137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7f159-46c0-4690-b262-fb0baf3471a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.668528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7f159-46c0-4690-b262-fb0baf3471a0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-e7f7f159-46c0-4690-b262-fb0baf3471a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:23.682528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:23.683657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:23.684521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:23.684521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:23.684521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:23.684521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 371/747] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:23.684750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202e237-de7d-4dca-a087-3c5c95f5069a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.685019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202e237-de7d-4dca-a087-3c5c95f5069a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.685336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202e237-de7d-4dca-a087-3c5c95f5069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.703740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9202e237-de7d-4dca-a087-3c5c95f5069a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:23.704909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9202e237-de7d-4dca-a087-3c5c95f5069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:23.765403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2b705-ab02-4500-99ab-f5b2491676c0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.781295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 372/748] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:23 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 12:19:23.781295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202e237-de7d-4dca-a087-3c5c95f5069a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.815843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.816973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.817416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.817798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.819490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:23.819886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.821448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.838284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.845234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 377/749] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 10 12:19:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:23.845234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.879378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202e237-de7d-4dca-a087-3c5c95f5069a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.879603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:23.916962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:23.917661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:23.917875 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:23.918140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:23.918864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:23.919161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:23.920612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:23.943386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 373/750] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:19:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:23.943386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:23.960874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418b84d-ba2d-4f1d-8040-4e536f7ca0db None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:23.999680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.000537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.000958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.001411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.002591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:24.003340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.005320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.018822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 378/751] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:24.024030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.048808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83290a40-3344-4e67-8207-61a28a361052 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.069533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.085974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.086577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.086900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.087171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.087803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.089154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.103604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 374/752] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:23 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:24.104503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.132560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.149098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e81b6-35be-4e61-9e74-c4f1129b4287 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.211954 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.212490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.212767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.213034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.245632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ee7d7-869c-48cc-b7df-7173827b70b5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.298553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.299190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.299462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.299725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.307921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.309846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.313961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.315375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:24.329944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:24.331285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:24.332187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:24.332187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:24.332187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:24.332187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 375/753] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:24.332187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.369201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.430648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.443024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e365a-850b-44c9-bd23-782e20df1f83 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.453875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 379/754] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:19:24.455304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.487459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.488106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.488535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.488883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.496399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.498119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:24.510652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:24.511578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:24.512549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:24.516694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 376/755] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:24.517748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.577194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882958e2-9d4b-4500-837a-192da1d120e6 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.627663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.628564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.628842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.629112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.630050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:24.630358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.630999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.634541 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.641696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406688a-e44b-46c7-9b64-d80042ab1683 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.651016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 380/756] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:24.651867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.685204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.686098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.686524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.686871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.700664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.702256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.713292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:24.719023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:24.720177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:24.721105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:24.721105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:24.721105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:24.722827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 377/757] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:24.723943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.781425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e68fd-4d18-4aa1-98a6-1036849a2c1d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.826932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.827526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.827918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.828226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.829017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:24.829430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.831527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.837382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581d737-b041-401c-bf80-fc3d931f4186 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.848200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 381/758] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:24.850142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.879865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.880966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.882090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.882617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.890289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.892030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:24.905071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:24.906043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:24.906916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:24.909084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 378/759] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:24.910340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:24.914738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.957781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9c443-f327-4864-b890-479c16cfd2ee None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:24.966105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:24.995441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:24.996040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:24.996314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:24.996581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:24.997222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:24.997551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:24.998917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:25.011361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 382/760] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:25.012240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:25.028977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd334a1-0fe3-4ccb-9fdb-e7a08bb55924 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:25.068011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:25.069064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.069574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.070049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.071117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=1fb6a74548184806972a7f8deb30f789) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:25.071690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:25.073577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:25.085467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 379/761] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:24 2022] GET /identity/v3/role_assignments?user.id=1fb6a74548184806972a7f8deb30f789&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:25.086747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:25.118783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd64db9-ad3d-4968-841c-8634c7e278cd None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:25.159590 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:25.160224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.160492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.160763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.161485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:25.161789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:25.163819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:25.194198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 383/762] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:25.198676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:25.223825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:25.265301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:25.280636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a433f-f992-4f95-96a3-c7c5690c0420 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:25.353567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:25.355008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.355509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.355985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.373229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:25.375462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:25.394414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:25.395357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:25.396243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:25.399306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 380/763] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:25.406903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:25.409942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bc300-b9b7-49ca-8fab-1fb680b522b1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:25.446205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:25.446751 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.447276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.447643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/536ca83c26354a77b1c5d263c57f4f7d/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.450157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=536ca83c26354a77b1c5d263c57f4f7d, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:25.451301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:25.468200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 384/764] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:25 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/536ca83c26354a77b1c5d263c57f4f7d/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:25.552005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e684-2e3f-4968-b741-0d9a23c33a27 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:25.590338 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:25.607427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd0ddf-0229-4794-b844-4e38673a3d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:25.608307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0ddf-0229-4794-b844-4e38673a3d80 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.608741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0ddf-0229-4794-b844-4e38673a3d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.609181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0ddf-0229-4794-b844-4e38673a3d80 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.610063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd0ddf-0229-4794-b844-4e38673a3d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:25.610520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd0ddf-0229-4794-b844-4e38673a3d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:25.612148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd0ddf-0229-4794-b844-4e38673a3d80 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:25.634445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 381/765] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:25.642363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:25.661328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b4bc-d56c-4042-9415-3295843c36b3 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.661844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b4bc-d56c-4042-9415-3295843c36b3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.662266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b4bc-d56c-4042-9415-3295843c36b3 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.664256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 382/766] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:25.670584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43be80-a72c-4e7c-ab0c-8b5a85868721 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.670872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43be80-a72c-4e7c-ab0c-8b5a85868721 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.671154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43be80-a72c-4e7c-ab0c-8b5a85868721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.695366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6c43be80-a72c-4e7c-ab0c-8b5a85868721 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:25.697073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6c43be80-a72c-4e7c-ab0c-8b5a85868721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:25.739773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:25.757589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde68519-053c-4e93-858f-39f9f81fd80b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:25.781496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 383/767] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:25.811389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:25.812006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:25.812290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:25.812567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:25.813328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=9c8a6dd668474aec9274ca29756be19d) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:25.813646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:25.815094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:25.823725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43be80-a72c-4e7c-ab0c-8b5a85868721 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:25.825335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 385/768] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:25 2022] GET /identity/v3/role_assignments?user.id=9c8a6dd668474aec9274ca29756be19d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:25.980945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43be80-a72c-4e7c-ab0c-8b5a85868721 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:25.999984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:26.018152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:26.019495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.019495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.019495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.020268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:26.021281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:26.031556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:26.032544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:26.033577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 384/769] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:25 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:26.041983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:26.161135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0374705-64a2-426b-bfbd-1795dcd701a7 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:26.207360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fb178-5e64-4a8c-b008-52f4b54e0124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:26.207969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fb178-5e64-4a8c-b008-52f4b54e0124 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.208252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fb178-5e64-4a8c-b008-52f4b54e0124 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.208526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fb178-5e64-4a8c-b008-52f4b54e0124 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.209166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fb178-5e64-4a8c-b008-52f4b54e0124 None admin] RBAC: Adding query filter params (name=network) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:26.210103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fb178-5e64-4a8c-b008-52f4b54e0124 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:26.211475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fb178-5e64-4a8c-b008-52f4b54e0124 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:26.220678 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 386/770] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:26.227000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:26.360294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6ea40-56e1-47d6-b342-dc33c3efacd5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:26.392852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d52af-7a00-4fb4-a83d-a492003f2541 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.393159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d52af-7a00-4fb4-a83d-a492003f2541 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.393452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d52af-7a00-4fb4-a83d-a492003f2541 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.394791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 387/771] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:26.408777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:26.408975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78febe81-fdcf-4e81-be80-80fddbc10512 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.409059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78febe81-fdcf-4e81-be80-80fddbc10512 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.409335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78febe81-fdcf-4e81-be80-80fddbc10512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.409450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.409688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.409957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.410591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:26.410895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:26.412400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:26.418387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 385/772] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:26.424902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:26.435675 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-78febe81-fdcf-4e81-be80-80fddbc10512 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:26.437669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-78febe81-fdcf-4e81-be80-80fddbc10512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:26.453781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:26.475860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:26.577352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4dc86-7249-4327-8013-0887696feba4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:26.581247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 388/773] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:26.621082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:26.621908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.622350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.622819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.623714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] RBAC: Adding query filter params (interface=public, service_id=2a7366519460475ba94f12aa392bf264, region_id=RegionOne) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:26.624175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:26.625723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:26.634573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 386/774] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:19:26 2022] GET /identity/v3/endpoints?service_id=2a7366519460475ba94f12aa392bf264&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:26.639641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78febe81-fdcf-4e81-be80-80fddbc10512 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:26.642233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:26.746144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78febe81-fdcf-4e81-be80-80fddbc10512 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:26.806656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23d38-7e9f-44d9-b340-acc0ac16b570 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:26.821610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:26.831887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:26.832623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.832805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.833083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.833712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:26.835638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:26.848015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:26.853134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:26.855131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:26.856106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:26.856927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:26.858115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:26.858535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:26.859343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:26.860852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:26.872672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 389/775] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 10 12:19:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:26.873055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 387/776] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:26 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:26.874575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:26.894212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:26.992656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da30e2-a843-455d-9896-ce261781c5c2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:27.029612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:27.030523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:27.030951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:27.031369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:27.032172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:27.032636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:27.034412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:27.041141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 390/777] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 10 12:19:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:27.047241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:27.156803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270d1e8-0c5d-4cb7-a13e-2714fee7acac None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:27.192930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:27.193852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:27.194281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:27.194697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:27.195467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:27.195944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:27.197482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:27.203279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 388/778] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 10 12:19:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:27.209169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:27.287748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:27.318465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4fce-f820-447b-b9ae-f66be043995d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:27.354285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707b13-1216-419c-b2a8-2ef93527c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:27.354895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707b13-1216-419c-b2a8-2ef93527c42f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:27.355175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707b13-1216-419c-b2a8-2ef93527c42f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:27.355448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707b13-1216-419c-b2a8-2ef93527c42f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:27.356151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707b13-1216-419c-b2a8-2ef93527c42f None admin] RBAC: Adding query filter params (interface=public, service_id=f08e1fdec29343c2bdbf8d952ef19086, region_id=RegionOne) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:27.356461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707b13-1216-419c-b2a8-2ef93527c42f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:27.357865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707b13-1216-419c-b2a8-2ef93527c42f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:27.365768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 391/779] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:19:27 2022] GET /identity/v3/endpoints?service_id=f08e1fdec29343c2bdbf8d952ef19086&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:27.371114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:27.422387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:27.495693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76326ce-0dc6-4ac2-8050-b7aa671f6be5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:27.543460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f70fc2-21d9-4d21-bcc0-6605fdfc05ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:27.544078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f70fc2-21d9-4d21-bcc0-6605fdfc05ff None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:27.544352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f70fc2-21d9-4d21-bcc0-6605fdfc05ff None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:27.544618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f70fc2-21d9-4d21-bcc0-6605fdfc05ff None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:27.545231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f70fc2-21d9-4d21-bcc0-6605fdfc05ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:27.546621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f70fc2-21d9-4d21-bcc0-6605fdfc05ff None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:27.552215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 389/780] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:27 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:27.704766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9134a-21fb-4ea3-84e5-008bc4198206 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:27.705781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9134a-21fb-4ea3-84e5-008bc4198206 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:27.706263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9134a-21fb-4ea3-84e5-008bc4198206 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:27.708219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 392/781] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:27.714230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6b98a-5f6f-400a-baf3-c84b11107ee8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:27.714816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6b98a-5f6f-400a-baf3-c84b11107ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:27.715298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6b98a-5f6f-400a-baf3-c84b11107ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:27.734698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6fd6b98a-5f6f-400a-baf3-c84b11107ee8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:27.736523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6fd6b98a-5f6f-400a-baf3-c84b11107ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:27.811086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 390/782] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:27.834914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9134a-21fb-4ea3-84e5-008bc4198206 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:27.971898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9134a-21fb-4ea3-84e5-008bc4198206 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:28.021659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60048d8a-f9f3-4a0d-b624-7bd45ee8483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:28.022697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60048d8a-f9f3-4a0d-b624-7bd45ee8483d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.023185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60048d8a-f9f3-4a0d-b624-7bd45ee8483d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.023663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60048d8a-f9f3-4a0d-b624-7bd45ee8483d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.029706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:28.042827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60048d8a-f9f3-4a0d-b624-7bd45ee8483d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-60048d8a-f9f3-4a0d-b624-7bd45ee8483d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:28.052313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:28.053908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:28.054772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:28.057383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 393/783] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:28.063799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6b98a-5f6f-400a-baf3-c84b11107ee8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:28.105044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:28.109070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea8447-7e43-44af-a90e-720a253665c0 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.109679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea8447-7e43-44af-a90e-720a253665c0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.110167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea8447-7e43-44af-a90e-720a253665c0 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.112228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 394/784] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:28.123618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44abbc5-ab76-43e4-a228-e89896d61724 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.123942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44abbc5-ab76-43e4-a228-e89896d61724 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.124175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44abbc5-ab76-43e4-a228-e89896d61724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.158417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e44abbc5-ab76-43e4-a228-e89896d61724 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:28.160708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e44abbc5-ab76-43e4-a228-e89896d61724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:28.237121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6b98a-5f6f-400a-baf3-c84b11107ee8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:28.297734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 395/785] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:28.322926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44abbc5-ab76-43e4-a228-e89896d61724 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:28.338865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:28.339828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.340310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.340809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.341734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:28.343787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:28.345567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:28.369103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 391/786] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:28.374456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:28.464027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44abbc5-ab76-43e4-a228-e89896d61724 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:28.491974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:28.492427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:28.505252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e504bf8-37ce-4b4b-bf17-748a7fd826ff None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:28.508362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6da63-9609-4c7c-a7be-98f0e9bebded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:28.509601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6da63-9609-4c7c-a7be-98f0e9bebded None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.510430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6da63-9609-4c7c-a7be-98f0e9bebded None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.513168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6da63-9609-4c7c-a7be-98f0e9bebded None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.543996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6da63-9609-4c7c-a7be-98f0e9bebded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-75e6da63-9609-4c7c-a7be-98f0e9bebded None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:28.550571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:28.551502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:28.552354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:28.557819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 396/787] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:28.557819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713bee5-83c2-43a5-b208-2e57114197bc None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.557819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713bee5-83c2-43a5-b208-2e57114197bc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.557819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713bee5-83c2-43a5-b208-2e57114197bc None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.561131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 397/788] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:28.561867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea330d-dae3-4fd8-8ed1-db88a4b576d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:28.562516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea330d-dae3-4fd8-8ed1-db88a4b576d0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.562835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea330d-dae3-4fd8-8ed1-db88a4b576d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.563106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea330d-dae3-4fd8-8ed1-db88a4b576d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.570641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713bee5-83c2-43a5-b208-2e57114197bc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:28.580762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea330d-dae3-4fd8-8ed1-db88a4b576d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:28.582269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea330d-dae3-4fd8-8ed1-db88a4b576d0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-baea330d-dae3-4fd8-8ed1-db88a4b576d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:28.611022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:28.611989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:28.612908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:28.612908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:28.612908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:28.617194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 392/789] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:28.618700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf451c-435d-4170-ba73-803c0bca2f83 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.622773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf451c-435d-4170-ba73-803c0bca2f83 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.623528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf451c-435d-4170-ba73-803c0bca2f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.642153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-0eaf451c-435d-4170-ba73-803c0bca2f83 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:28.643662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0eaf451c-435d-4170-ba73-803c0bca2f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:28.755761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 393/790] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:28.759510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf451c-435d-4170-ba73-803c0bca2f83 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:28.786704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713bee5-83c2-43a5-b208-2e57114197bc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:28.830519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:28.831125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.831412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.831672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.832501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:28.832816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:28.835340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:28.850125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 398/791] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:28.850125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:28.894328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:28.909191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:28.920484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf451c-435d-4170-ba73-803c0bca2f83 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:28.964106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:28.964725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:28.965005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:28.965289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:28.966005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:28.966251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:28.967635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:28.986684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b242d90-915f-4d44-9efd-22cd168545d5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:28.989417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 394/792] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:28.990502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.033995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.034561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.034840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.035110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.050059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:29.057117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:29.058228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:29.059073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:29.061473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 399/793] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:29.062517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.123847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbad5a-a8fd-421e-86f9-4be161b2731a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.170285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.170932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.171189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.171458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.172719 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd066ac-430b-406b-9d65-7ad27826c8c7 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.174333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.184610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.187699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.189898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:29.209475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:29.210403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:29.211286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:29.211286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:29.211286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:29.217807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 395/794] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:29.217807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.218967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.219595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.219869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.220134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.230174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.231615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:29.246243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:29.247184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:29.248764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:29.252163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 400/795] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:29.252980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.355181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec60a9-f17e-4361-877e-e461cb5eb829 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.376821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd5a7a-6dad-4684-b008-3a4aa8504a24 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.392727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.394000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.394522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.394996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.395866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:29.396370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.398033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.407044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 396/796] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:29.408098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.422433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.423038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.423314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.423582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.424223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:29.424529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.425986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.441659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 401/797] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:29.441659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.464654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.471193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.521912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f704-ad78-4f58-9971-f2efe0752c36 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.554494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130fb90-cf72-4f7b-99a7-2f3670f131ab None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.557528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3d19b-7d45-48b6-8a88-5bda29ed1285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.558141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d19b-7d45-48b6-8a88-5bda29ed1285 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.558422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d19b-7d45-48b6-8a88-5bda29ed1285 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.558695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d19b-7d45-48b6-8a88-5bda29ed1285 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.559429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3d19b-7d45-48b6-8a88-5bda29ed1285 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:29.559735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3d19b-7d45-48b6-8a88-5bda29ed1285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.561162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3d19b-7d45-48b6-8a88-5bda29ed1285 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.587879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 397/798] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:29.587879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e030-ef37-40c5-9134-3b6d0475ab82 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.587879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e030-ef37-40c5-9134-3b6d0475ab82 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.587879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e030-ef37-40c5-9134-3b6d0475ab82 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.587879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 398/799] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:29.587879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56e030-ef37-40c5-9134-3b6d0475ab82 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.599066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.599699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.600507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.601327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.612672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.614101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:29.629570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:29.630622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:29.631502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:29.631502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:29.631502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:29.631502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 402/800] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:29.631722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.706396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56e030-ef37-40c5-9134-3b6d0475ab82 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.715940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.718142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.736670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078675e-abdb-4465-bfda-a4f5fbd5eaa7 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.747976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28a8a-31b5-4892-b99e-75334a72b745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.748686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28a8a-31b5-4892-b99e-75334a72b745 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.749067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28a8a-31b5-4892-b99e-75334a72b745 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.749485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28a8a-31b5-4892-b99e-75334a72b745 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.757165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c28a8a-31b5-4892-b99e-75334a72b745 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.758731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c28a8a-31b5-4892-b99e-75334a72b745 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.770322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-79c28a8a-31b5-4892-b99e-75334a72b745 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:29.770892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:29.772012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:29.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:29.773863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.773863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.773863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.774104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 399/801] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:29.774424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e2dad-2484-4ece-8ba1-d6423a1fb8e5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.774703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e2dad-2484-4ece-8ba1-d6423a1fb8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.774971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e2dad-2484-4ece-8ba1-d6423a1fb8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.788697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.790187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.794954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-245e2dad-2484-4ece-8ba1-d6423a1fb8e5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:29.796249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-245e2dad-2484-4ece-8ba1-d6423a1fb8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:29.805550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:29.806542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:29.807448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:29.807448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:29.807448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:29.807448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 403/802] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:29.807448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.879092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 400/803] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:29.879092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e2dad-2484-4ece-8ba1-d6423a1fb8e5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.914278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a2788-fed9-434a-91bc-2c5c3e39690c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:29.944653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.950074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:29.950680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:29.950960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:29.951232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:29.951871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:29.952182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:29.953667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:29.965293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 404/804] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:29.966073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:29.979342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:29.983474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e2dad-2484-4ece-8ba1-d6423a1fb8e5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.016903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.017534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.017810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.018078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.018795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:30.019096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.020505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.038478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 401/805] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:30.039260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.067613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee986-28f8-42cb-bfa6-c91ae6ec7449 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.101021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92de6c3-c73e-476a-be12-11304b180271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.101653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92de6c3-c73e-476a-be12-11304b180271 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.101937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92de6c3-c73e-476a-be12-11304b180271 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.102211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92de6c3-c73e-476a-be12-11304b180271 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.103097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92de6c3-c73e-476a-be12-11304b180271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:30.103408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92de6c3-c73e-476a-be12-11304b180271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.104776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92de6c3-c73e-476a-be12-11304b180271 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.116529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 405/806] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:30.117781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e361c-b102-4e2d-becc-91315f46a323 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.118070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e361c-b102-4e2d-becc-91315f46a323 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.118349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e361c-b102-4e2d-becc-91315f46a323 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.119685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 406/807] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:30.120299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e361c-b102-4e2d-becc-91315f46a323 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.137346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217729a7-7cb5-4bfe-a93e-231eb3452e6e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.171042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.171664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.171944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.172218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] PATH_INFO: `/v3/services/network` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.172800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.174201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.174440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:30.182933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:30.184137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:30.184975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 402/808] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/services/network => generated 88 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:30.185823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.210009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.225510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e361c-b102-4e2d-becc-91315f46a323 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.259142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.259765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.260042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.260311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.268653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:30.280859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:30.282092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 407/809] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:30.283044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.291325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174faf2-5eda-4629-a481-af9a041f8972 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.325449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.326056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.326364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.326607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.381593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bd669-562f-4ce9-b6c2-cd3eb26ee1f3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.393837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.394292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.395271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.415790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.416333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d167de8-73f1-4f02-a48a-186228d88470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.417031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167de8-73f1-4f02-a48a-186228d88470 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.417419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167de8-73f1-4f02-a48a-186228d88470 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.417733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167de8-73f1-4f02-a48a-186228d88470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.424370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d167de8-73f1-4f02-a48a-186228d88470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.425848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d167de8-73f1-4f02-a48a-186228d88470 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-2d167de8-73f1-4f02-a48a-186228d88470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:30.436841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:30.437921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:30.438849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 408/810] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:30.439555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d209-7615-4eb6-ac35-e1ed4cd8af3f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.439555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d209-7615-4eb6-ac35-e1ed4cd8af3f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.439679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d209-7615-4eb6-ac35-e1ed4cd8af3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.457892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-7c42d209-7615-4eb6-ac35-e1ed4cd8af3f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:30.459089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7c42d209-7615-4eb6-ac35-e1ed4cd8af3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:30.497429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 403/811] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:19:30.498262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.525331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 409/812] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:30.526245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42d209-7615-4eb6-ac35-e1ed4cd8af3f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.600369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b21c-55fa-49aa-88fa-28135f0da21f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.620584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.635605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.636208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.636328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42d209-7615-4eb6-ac35-e1ed4cd8af3f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.636489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.636761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.637419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:30.637735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.637879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.639188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.643576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 404/813] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:30.644400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.675035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.675639 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.675915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.676185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.676918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:30.677222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.679188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.697214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 410/814] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:30.698033 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.742261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7062c-2e42-46bf-a02c-806fb2a2e3d2 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.775211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.775810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.776083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.776348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.777026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:30.777355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.778776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.790012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 405/815] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:30.790622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.798252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3861d-59a3-4cba-8734-c5b02598873b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.832648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.833299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.833587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.833861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.835698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.844045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.845515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:30.859951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:30.860965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:30.861887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:30.861887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:30.861887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:30.861887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 411/816] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:30.861887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.876995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:30.890940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdb121-a3ee-43a7-960c-6679f014c428 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.923926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.924545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.924820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.925090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.925686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.927098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:30.935938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:30.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:30.937932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 406/817] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:30.938725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:30.963133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2c2c-b635-4ff8-8f94-1b082e5888db None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:30.995934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:30.996532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:30.996811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:30.997084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:30.997748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] RBAC: Adding query filter params (type=network) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:30.998052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:30.999419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.004459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 412/818] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:31.005201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.035054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb704e-2355-4e94-93c2-e58719db52bc None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.044159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.069254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.069892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.070176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.070443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.080924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.081333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.082389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:31.096474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:31.097463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:31.098385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:31.098385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:31.098385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:31.098385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 407/819] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:31.098385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.109512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581cf40-c43c-42b8-bc89-94eb30e2c008 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.141983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.142626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.142906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.143178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/1fb6a74548184806972a7f8deb30f789/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.145034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=1fb6a74548184806972a7f8deb30f789, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.146500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.159050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 413/820] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:31 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/1fb6a74548184806972a7f8deb30f789/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:31.159815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.202741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183baf-fa04-4938-847b-5c97dd7c869e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.241514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.242257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.242537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.242806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.243502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:31.243808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.245219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.263148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 408/821] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:31.263148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.279058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276a801-c327-42a6-baae-8fb673b9809e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.283750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.323252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.323867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.324265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.324739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.325933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:31.326243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.327640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.329627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.334170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 414/822] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:31.335007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.378564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18c3d8-237a-4d80-bd6d-e0b0c1ca2ed2 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.413842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.414481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.414831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.415016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.415673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.417044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.440173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 409/823] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:19:31 2022] POST /identity/v3/endpoints => generated 337 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:31.441063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.464203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6aa0f-f2f2-481a-8a3c-b8731f315621 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.524536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.525499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.525886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.526270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.527156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:31.527570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.529132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.546768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.549240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 415/824] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:31.550244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.581313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d20af0-c106-4f5a-8f75-e3ea22f0c198 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.620777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.627442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.628080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.628356 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.628628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.635739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.637177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:31.652019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:31.652954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:31.653892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 410/825] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:31.654380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.675546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23005e3-4a39-4213-8f0d-e68327e3de27 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.723982 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.724712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.725646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.725918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.726547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:31.726848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.728488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.737371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 416/826] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:31.737371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.779604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a35850-203d-40f1-ac99-22f4db6d48ff None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.814781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.815412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.815691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.815960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.819428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.824967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.826752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:31.839031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:31.840100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:31.841177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:31.842855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 411/827] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:31.843763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.849554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f409a8-fc33-4d0d-8a1e-7b8db587370a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.871692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:31.887109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.887725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.888028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.888503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.888967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:31.889293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.890735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:31.904343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 417/828] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:31.904988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:31.956313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e0e07-fef2-4577-8f04-07a240417396 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:31.991096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:31.991913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:31.992370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:31.992643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:31.993325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:31.994689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:32.008216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019e90e-2403-4f81-a0a9-4aca136235cb None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:32.021510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 412/829] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:19:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:32.021510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:32.051343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:32.052006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:32.052288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:32.052560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:32.053376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:32.053687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:32.055147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:32.069296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 418/830] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:32.078370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:32.097642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:32.125997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:32.162151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1d76f-5ab2-4049-a229-1102b97421f1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:32.207729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b00296-fe60-4e52-8954-9da63e04d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:32.208489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b00296-fe60-4e52-8954-9da63e04d40c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:32.208917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b00296-fe60-4e52-8954-9da63e04d40c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:32.209334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b00296-fe60-4e52-8954-9da63e04d40c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:32.210163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b00296-fe60-4e52-8954-9da63e04d40c None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=536ca83c26354a77b1c5d263c57f4f7d) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:32.210998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b00296-fe60-4e52-8954-9da63e04d40c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:32.212990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b00296-fe60-4e52-8954-9da63e04d40c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:32.214218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae0e64-9050-4b4c-b5d8-77a0dd7df0bf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:32.219450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 413/831] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:32 2022] GET /identity/v3/role_assignments?user.id=536ca83c26354a77b1c5d263c57f4f7d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:32.256820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5486bc-3119-4bd9-8f4d-a3081bfd2102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:32.257493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5486bc-3119-4bd9-8f4d-a3081bfd2102 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:32.257773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5486bc-3119-4bd9-8f4d-a3081bfd2102 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:32.258042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5486bc-3119-4bd9-8f4d-a3081bfd2102 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/9c8a6dd668474aec9274ca29756be19d/roles/8c3fb9b334554c2c831b5e08ab522856` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:32.276637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5486bc-3119-4bd9-8f4d-a3081bfd2102 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=9c8a6dd668474aec9274ca29756be19d, role_id=8c3fb9b334554c2c831b5e08ab522856)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:32.278232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5486bc-3119-4bd9-8f4d-a3081bfd2102 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:32.296216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 419/832] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:32 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/9c8a6dd668474aec9274ca29756be19d/roles/8c3fb9b334554c2c831b5e08ab522856 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:33.186575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf2ba-965d-4972-a4b8-4b2960df329c None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:33.187107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf2ba-965d-4972-a4b8-4b2960df329c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:33.187107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf2ba-965d-4972-a4b8-4b2960df329c None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:33.188714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 414/833] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:33.194728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd148-d476-40de-aaad-b2362da0409c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:33.195146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd148-d476-40de-aaad-b2362da0409c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:33.195286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebd148-d476-40de-aaad-b2362da0409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:33.213323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-75ebd148-d476-40de-aaad-b2362da0409c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:33.214519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-75ebd148-d476-40de-aaad-b2362da0409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:33.287948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 420/834] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:33.310919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bf2ba-965d-4972-a4b8-4b2960df329c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:33.417973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bf2ba-965d-4972-a4b8-4b2960df329c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:33.435895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:33.455152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:33.455693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:33.455965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:33.456227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:33.467061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:33.471663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:33.472589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:33.473433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 415/835] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:33.479822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd148-d476-40de-aaad-b2362da0409c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:33.497877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:33.585242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebd148-d476-40de-aaad-b2362da0409c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:33.621149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:33.621791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:33.622070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:33.622341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:33.623000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:33.623301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:33.624671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:33.633458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 421/836] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:33.639013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:33.750530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c0ef-2d1e-45af-8748-9a8dfeccb48c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:33.784566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:33.785193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:33.785497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:33.785766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:33.797134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:33.798695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:33.813032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:33.814003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:33.814881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:33.814881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:33.814881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:33.814881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 416/837] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:33.820133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:33.904223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:33.929136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0fd6e-c090-42de-89e8-4876567b95cf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:33.965020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:33.965657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:33.965936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:33.966207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:33.966850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:33.967157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:33.968529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:33.986552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 422/838] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:33.987886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.017753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:34.095531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a50d85-8e56-47ad-a5de-ee5a2d4b5325 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:34.129952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82daac67-9911-4b94-b941-f2f03df5a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:34.130578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daac67-9911-4b94-b941-f2f03df5a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.130855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daac67-9911-4b94-b941-f2f03df5a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.131615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daac67-9911-4b94-b941-f2f03df5a7b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.139290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82daac67-9911-4b94-b941-f2f03df5a7b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:34.140738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82daac67-9911-4b94-b941-f2f03df5a7b5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-82daac67-9911-4b94-b941-f2f03df5a7b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:34.152526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:34.153491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:34.154496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 417/839] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:19:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:34.159775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.215542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65cbe-23f4-4fb1-9fe8-e3c68bf350e9 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.216194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65cbe-23f4-4fb1-9fe8-e3c68bf350e9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.216194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65cbe-23f4-4fb1-9fe8-e3c68bf350e9 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.218872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 418/840] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:34.223605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa38293-66bc-409b-a07f-1381b63f71af None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.224109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa38293-66bc-409b-a07f-1381b63f71af None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.224578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa38293-66bc-409b-a07f-1381b63f71af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.242981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-bfa38293-66bc-409b-a07f-1381b63f71af None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:34.244489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bfa38293-66bc-409b-a07f-1381b63f71af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:34.274247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141fe1c-e866-4e2b-98d4-8cb4a8d447ce None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:34.316999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:34.317642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.317920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.318194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.319448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:34.319756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:34.325467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:34.325760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 419/841] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:34.329701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:34.341025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 423/842] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:34.344454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa38293-66bc-409b-a07f-1381b63f71af None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.346227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.364757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:34.451527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb136d-337d-4b9a-a2fa-e8ac03a675ba None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:34.453746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa38293-66bc-409b-a07f-1381b63f71af None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:34.485732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68d96a-a6ff-4fff-9b11-fde8de00cc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:34.486327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68d96a-a6ff-4fff-9b11-fde8de00cc12 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.486607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68d96a-a6ff-4fff-9b11-fde8de00cc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.486883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68d96a-a6ff-4fff-9b11-fde8de00cc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.488107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36726923-d1c3-4691-a851-c5a8f5855b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:34.488723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36726923-d1c3-4691-a851-c5a8f5855b9e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.489001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36726923-d1c3-4691-a851-c5a8f5855b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.489288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36726923-d1c3-4691-a851-c5a8f5855b9e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.489908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36726923-d1c3-4691-a851-c5a8f5855b9e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:34.491321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36726923-d1c3-4691-a851-c5a8f5855b9e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-36726923-d1c3-4691-a851-c5a8f5855b9e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:34.499726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:34.500699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:34.506510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 420/843] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:34.507294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb0c19-a3dc-4975-9913-bbe4c0016d00 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.507294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb0c19-a3dc-4975-9913-bbe4c0016d00 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.507294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb0c19-a3dc-4975-9913-bbe4c0016d00 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.507294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 421/844] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:34.507294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d67ea5-dd8b-42ca-8101-2debca55cdf5 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.507294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d67ea5-dd8b-42ca-8101-2debca55cdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.507648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d67ea5-dd8b-42ca-8101-2debca55cdf5 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.507648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 422/845] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:34.507648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d67ea5-dd8b-42ca-8101-2debca55cdf5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.552849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:34.559942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68d96a-a6ff-4fff-9b11-fde8de00cc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:34.561421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68d96a-a6ff-4fff-9b11-fde8de00cc12 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:34.589683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:34.609406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d67ea5-dd8b-42ca-8101-2debca55cdf5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:34.643565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf27dc-2c0c-4dfc-8b0b-68dd84127dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:34.644167 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf27dc-2c0c-4dfc-8b0b-68dd84127dde None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.644444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf27dc-2c0c-4dfc-8b0b-68dd84127dde None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.644713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf27dc-2c0c-4dfc-8b0b-68dd84127dde None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.645361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf27dc-2c0c-4dfc-8b0b-68dd84127dde None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:34.645665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf27dc-2c0c-4dfc-8b0b-68dd84127dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:34.647545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf27dc-2c0c-4dfc-8b0b-68dd84127dde None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:34.651893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 423/846] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:34.653002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b2b56-9a8f-4985-9c16-5eb5d58a48f4 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.653309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b2b56-9a8f-4985-9c16-5eb5d58a48f4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.653590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b2b56-9a8f-4985-9c16-5eb5d58a48f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.668017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 424/847] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:19:34.668017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62872a05-8d36-411f-a912-d2a6f84afac1 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.668017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62872a05-8d36-411f-a912-d2a6f84afac1 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.668017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62872a05-8d36-411f-a912-d2a6f84afac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.673761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-137b2b56-9a8f-4985-9c16-5eb5d58a48f4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:34.674939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-137b2b56-9a8f-4985-9c16-5eb5d58a48f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:34.682912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-62872a05-8d36-411f-a912-d2a6f84afac1 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:34.684066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-62872a05-8d36-411f-a912-d2a6f84afac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:34.741646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 424/848] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:34.742484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b2b56-9a8f-4985-9c16-5eb5d58a48f4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.761288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 425/849] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:34.761288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62872a05-8d36-411f-a912-d2a6f84afac1 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.779991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:34.831310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:34.854016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62872a05-8d36-411f-a912-d2a6f84afac1 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:34.854527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b2b56-9a8f-4985-9c16-5eb5d58a48f4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:34.887582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:34.887753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:34.888189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.888284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:34.888464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.888562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:34.888732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.888819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:34.889410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:34.889710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:34.891073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:34.895552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 425/850] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:34.896130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:34.905898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:34.907408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:34.923869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:34.925030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:34.925978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:34.925978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:34.925978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:34.925978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 426/851] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:34.926203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.019682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.027773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5ed38-f0ee-4c4e-9696-7ef49b2a514d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.044548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0c69e-c8e8-4560-9310-a3928e74df4e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.065299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.067192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.067192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.067192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.067192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.068929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:35.078213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:35.079183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:35.080603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 426/852] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:35.081943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.085674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.089745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.090462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.090810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.091167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.102762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:35.108077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:35.109045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 427/853] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:19:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:35.109962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.201148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402dee5-a72c-4873-b807-7b0452f7d329 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.214338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17290efc-8090-4b93-9443-85da46a02d8b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.238240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.239414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.239698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.239972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.241138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:35.241479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.242855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.250204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.250813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.251092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.251365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.251996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:35.252302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.253675 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.257596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 427/854] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:35.258694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.264049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 428/855] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:35.264922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.270936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.327271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.375291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d92877-d184-40e6-9b42-e687d323b546 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.382883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463889-96e4-4852-8241-d6149ba9aa56 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.408622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.409291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.409584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.409855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.410520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:35.410819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.412832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.420889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 428/856] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:35.421703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.421952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.422788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.423071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.423340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.432354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.433983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:35.446162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:35.447043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:35.447910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 429/857] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:35.448392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.510648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.537993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba763eb4-78ef-4aa6-877b-074c6a4e40fe None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.551484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce1e63-0ecb-4538-86f5-01ee2ec24add None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.563481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.579168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.579862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.580054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.580324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.581174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:35.581311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.582648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.589747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.590388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.590671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.590943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.595212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 429/858] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:35.595212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.608186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.609779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:35.627115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:35.628102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:35.629012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:35.629012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:35.629012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:35.629012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 430/859] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:35.634248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.740153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdefa8e-92d7-4143-9812-bff0ecf22c12 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.749878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3672bde-b25d-42e7-a0dd-4e6824ae7e2b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.765711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.784512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.785358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.785696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.786045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.786879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:35.787297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.789199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.791529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.792810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.793309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.793788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.794615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:35.795098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.796651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.803371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 430/860] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:35.811297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 431/861] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:35.811552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.816367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:35.870996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:35.925706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec074f-9d33-4d60-9476-6910cb67ca3a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.929472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad212-2959-4d29-b3d3-8222b7378200 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:35.963532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.964488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.964960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.965565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.966534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=1fb6a74548184806972a7f8deb30f789) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:35.967032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:35.967154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.967766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:35.968121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:35.968467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:35.969333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.980395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:35.980395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:35.980780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 431/862] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/role_assignments?user.id=1fb6a74548184806972a7f8deb30f789&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:35.992352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:35.993385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:35.994316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:35.998800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 432/863] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:36.000357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:36.129534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a134772-7830-4b4f-b95d-77e9deafb5a7 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:36.152567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:36.173888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:36.174511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:36.174791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:36.175063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:36.175856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:36.176109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:36.177567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:36.197429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 432/864] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 10 12:19:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:36.202800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:36.246101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:36.316981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9de39a-5fbf-4b1b-863b-4b7e6b461248 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:36.354332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:36.354950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:36.355230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:36.355505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:36.427185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:36.428600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:36.463608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:36.532114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 433/865] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:19:36.537804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:36.649645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b951f7-bc3d-4798-9c2a-301cbaedc271 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:36.685079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:36.685666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:36.685942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:36.686256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:36.697180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:36.698667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:36.713742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:36.714727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:36.715613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:36.715613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:36.715613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:36.715613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 433/866] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:36.721865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:36.830739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7319d-493b-4491-a9e5-7dbe102c74c0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:36.865171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6756f-3cd1-47c8-92d5-fe52bd5f12f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:36.866311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6756f-3cd1-47c8-92d5-fe52bd5f12f4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:36.866588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6756f-3cd1-47c8-92d5-fe52bd5f12f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:36.866859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6756f-3cd1-47c8-92d5-fe52bd5f12f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:36.867493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6756f-3cd1-47c8-92d5-fe52bd5f12f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:36.867794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6756f-3cd1-47c8-92d5-fe52bd5f12f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:36.869162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6756f-3cd1-47c8-92d5-fe52bd5f12f4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:36.878094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:36.882243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 434/867] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:36.890335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:36.917417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d4b2-ef12-46c0-9793-f4903bb0bdba None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:36.918042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d4b2-ef12-46c0-9793-f4903bb0bdba None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:36.918520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d4b2-ef12-46c0-9793-f4903bb0bdba None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:36.919424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:36.920527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 435/868] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:36.925456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d045-a8eb-4fdb-82d0-5f74154e7034 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:36.926228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d045-a8eb-4fdb-82d0-5f74154e7034 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:36.926733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d045-a8eb-4fdb-82d0-5f74154e7034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:36.946264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-c1a7d045-a8eb-4fdb-82d0-5f74154e7034 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:36.947781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c1a7d045-a8eb-4fdb-82d0-5f74154e7034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:37.010180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be9611-c2eb-4378-8172-057e43d66d86 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:37.041355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 436/869] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:37.053747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:37.054616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:37.055115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:37.055601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:37.062758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:37.064187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d045-a8eb-4fdb-82d0-5f74154e7034 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:37.066901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:37.079813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:37.080742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:37.081646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:37.085945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 434/870] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:37.086809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:37.181220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d045-a8eb-4fdb-82d0-5f74154e7034 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:37.181593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:37.198552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260bf6e-3e1d-4ab6-839e-ada8725afda0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:37.217756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:37.222668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:37.224082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:37.224372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:37.225658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:37.226738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:37.229079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:37.235932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a792d2-f8b4-4ab1-8928-1e81c67919d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:37.236613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a792d2-f8b4-4ab1-8928-1e81c67919d1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:37.236820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a792d2-f8b4-4ab1-8928-1e81c67919d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:37.237127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a792d2-f8b4-4ab1-8928-1e81c67919d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:37.238056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a792d2-f8b4-4ab1-8928-1e81c67919d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:37.238354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a792d2-f8b4-4ab1-8928-1e81c67919d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:37.239818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a792d2-f8b4-4ab1-8928-1e81c67919d1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:37.250235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 437/871] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:37 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:19:37.261085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 435/872] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:37.267231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:37.398364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cd6f6-5aa9-4667-b630-60c450d765a3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:37.436888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862ac6-9ec7-4d9c-a221-7c3dda923d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:37.437555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862ac6-9ec7-4d9c-a221-7c3dda923d7c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:37.437794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862ac6-9ec7-4d9c-a221-7c3dda923d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:37.438057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862ac6-9ec7-4d9c-a221-7c3dda923d7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:37.438782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862ac6-9ec7-4d9c-a221-7c3dda923d7c None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=9c8a6dd668474aec9274ca29756be19d) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:37.439136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862ac6-9ec7-4d9c-a221-7c3dda923d7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:37.440579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862ac6-9ec7-4d9c-a221-7c3dda923d7c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:37.446867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 438/873] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:37 2022] GET /identity/v3/role_assignments?user.id=9c8a6dd668474aec9274ca29756be19d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:37.715100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0223b8-fd13-4394-9477-750e495e800a None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:37.715634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0223b8-fd13-4394-9477-750e495e800a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:37.715634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0223b8-fd13-4394-9477-750e495e800a None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:37.717217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 436/874] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:37.722561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5cce8-1583-470b-b4bf-a07e7d9b6b24 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:37.722844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5cce8-1583-470b-b4bf-a07e7d9b6b24 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:37.723118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5cce8-1583-470b-b4bf-a07e7d9b6b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:37.742857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1cf5cce8-1583-470b-b4bf-a07e7d9b6b24 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:37.744043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1cf5cce8-1583-470b-b4bf-a07e7d9b6b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:37.789505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:37.823175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 439/875] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:37.840894 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0223b8-fd13-4394-9477-750e495e800a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:37.948647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0223b8-fd13-4394-9477-750e495e800a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:37.985003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e88572-f3ea-47b3-89cf-1dc9b4c917e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:37.985643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88572-f3ea-47b3-89cf-1dc9b4c917e8 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:37.985919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88572-f3ea-47b3-89cf-1dc9b4c917e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:37.986185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88572-f3ea-47b3-89cf-1dc9b4c917e8 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:37.986845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e88572-f3ea-47b3-89cf-1dc9b4c917e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:37.988210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e88572-f3ea-47b3-89cf-1dc9b4c917e8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:38.006549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 437/876] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:37 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:38.125445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8f81d-a81f-41d7-a9c4-6249ec5085d4 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:38.125841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8f81d-a81f-41d7-a9c4-6249ec5085d4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:38.126032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8f81d-a81f-41d7-a9c4-6249ec5085d4 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:38.127703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 440/877] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:38.133445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3209-5f30-4e06-8005-9233e4e05406 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:38.133730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3209-5f30-4e06-8005-9233e4e05406 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:38.134007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3209-5f30-4e06-8005-9233e4e05406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:38.142519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:38.160122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-97af3209-5f30-4e06-8005-9233e4e05406 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:38.162073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-97af3209-5f30-4e06-8005-9233e4e05406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:38.242696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 438/878] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:38.264344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8f81d-a81f-41d7-a9c4-6249ec5085d4 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:38.375646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8f81d-a81f-41d7-a9c4-6249ec5085d4 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:38.410950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:38.411822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:38.412835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:38.413256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:38.414014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:38.415559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:38.424526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:38.425761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:38.426795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:38.433373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 441/879] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:38.435391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af3209-5f30-4e06-8005-9233e4e05406 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:38.545052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af3209-5f30-4e06-8005-9233e4e05406 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:38.585645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:38.592113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b1828-799d-4fb2-a042-bb1cbe6604a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:38.592720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b1828-799d-4fb2-a042-bb1cbe6604a1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:38.592998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b1828-799d-4fb2-a042-bb1cbe6604a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:38.593260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b1828-799d-4fb2-a042-bb1cbe6604a1 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:38.596585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b1828-799d-4fb2-a042-bb1cbe6604a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:38.596585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b1828-799d-4fb2-a042-bb1cbe6604a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:38.596585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b1828-799d-4fb2-a042-bb1cbe6604a1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:38.601845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 439/880] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:38.607771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:38.654077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:38.718883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ffa27-5f6f-4111-8ff3-6d283fdec5f3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:38.753814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d82734-d8ce-4810-a93c-6aa8b5168edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:38.754689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d82734-d8ce-4810-a93c-6aa8b5168edd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:38.754969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d82734-d8ce-4810-a93c-6aa8b5168edd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:38.755237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d82734-d8ce-4810-a93c-6aa8b5168edd None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:38.756398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d82734-d8ce-4810-a93c-6aa8b5168edd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:38.756714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d82734-d8ce-4810-a93c-6aa8b5168edd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:38.758112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d82734-d8ce-4810-a93c-6aa8b5168edd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:38.763017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 442/881] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:39.353203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9461e7a-0497-4cc6-bba9-5289afcd866b None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.353743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9461e7a-0497-4cc6-bba9-5289afcd866b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.353854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9461e7a-0497-4cc6-bba9-5289afcd866b None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.355358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 440/882] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:39.360584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e78a06-f841-41cc-88bc-f7c2d3ffc4bf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.360877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e78a06-f841-41cc-88bc-f7c2d3ffc4bf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.361154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e78a06-f841-41cc-88bc-f7c2d3ffc4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.379528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-25e78a06-f841-41cc-88bc-f7c2d3ffc4bf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:39.380724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-25e78a06-f841-41cc-88bc-f7c2d3ffc4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:39.457107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 443/883] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:39.479628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9461e7a-0497-4cc6-bba9-5289afcd866b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:39.549007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c12e8-60cb-4820-83df-0acefc104412 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.549342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c12e8-60cb-4820-83df-0acefc104412 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.549600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c12e8-60cb-4820-83df-0acefc104412 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.552051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 444/884] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:39.556855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ad8ce-0d9e-4652-9b94-ccc50286c2c2 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.557129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ad8ce-0d9e-4652-9b94-ccc50286c2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.557432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ad8ce-0d9e-4652-9b94-ccc50286c2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.577601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-302ad8ce-0d9e-4652-9b94-ccc50286c2c2 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:39.578887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-302ad8ce-0d9e-4652-9b94-ccc50286c2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:39.587570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9461e7a-0497-4cc6-bba9-5289afcd866b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:39.608336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:39.624535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:39.625166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.625471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.625742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.626327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:39.628103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:39.639652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:39.640630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:39.641524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 441/885] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:39.647345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:39.666805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 445/886] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:39.687146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ad8ce-0d9e-4652-9b94-ccc50286c2c2 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:39.706306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:39.750228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cec744-3136-4df0-9137-d53c52e981c4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:39.785355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:39.785968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.786246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.786517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.787206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:39.788006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:39.789821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:39.797339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 442/887] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:39.801196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:39.810121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ad8ce-0d9e-4652-9b94-ccc50286c2c2 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:39.856495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:39.857562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.858047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.858955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.873187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:39.879521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:39.880430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:39.881332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:39.883995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 446/888] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 10 12:19:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:39.890234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:39.919950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:39.923532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05c259-b2be-4a42-8caa-41ad326c82ad None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:39.950824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:39.964536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:39.965242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:39.965623 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:39.965980 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:39.966683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:39.967045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:39.968871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:39.979575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 443/889] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:39.980741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.022564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11759be4-3801-4aa4-bf4c-422b11f55189 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.059528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.060293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.060677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.061018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.061870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:40.062240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.063767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.077569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 447/890] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 10 12:19:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:40.079119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.097966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d24296-2079-4aba-9121-ccb12961782b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.133295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.133931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.134520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.134631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.135412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] RBAC: Adding query filter params (interface=public, service_id=4b70e2d4b497446ea3f612959a78ec49, region_id=RegionOne) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:40.135786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.137213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.147925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 444/891] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:19:39 2022] GET /identity/v3/endpoints?service_id=4b70e2d4b497446ea3f612959a78ec49&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:40.148860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.186606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:40.202218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:40.208510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c815ab-27a3-4366-b623-c3ffe643ee6c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.254251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017e19d-22e2-495d-b28b-dd709a1f032a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.255420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017e19d-22e2-495d-b28b-dd709a1f032a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.256418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017e19d-22e2-495d-b28b-dd709a1f032a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.256906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017e19d-22e2-495d-b28b-dd709a1f032a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.257389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942b1e-794c-4c1f-b265-118be56a8aeb None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.270275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017e19d-22e2-495d-b28b-dd709a1f032a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.272374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017e19d-22e2-495d-b28b-dd709a1f032a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-9017e19d-22e2-495d-b28b-dd709a1f032a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:40.287240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:40.288195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:40.289111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:40.289111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:40.289111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:40.289111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 448/892] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:40.289924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eec53-54a7-45a3-b1fa-2a57bac36c46 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.289924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eec53-54a7-45a3-b1fa-2a57bac36c46 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.290101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87eec53-54a7-45a3-b1fa-2a57bac36c46 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.293394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 449/893] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:40.294953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eec53-54a7-45a3-b1fa-2a57bac36c46 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.302878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b594f-300b-4e03-875b-eb18ad414581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.303559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b594f-300b-4e03-875b-eb18ad414581 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.303914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b594f-300b-4e03-875b-eb18ad414581 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.304250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b594f-300b-4e03-875b-eb18ad414581 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.304916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b594f-300b-4e03-875b-eb18ad414581 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.306771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b594f-300b-4e03-875b-eb18ad414581 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.321308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 445/894] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:19:40.321308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4343a-17bf-429b-aaed-a645930b9eee None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.321308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4343a-17bf-429b-aaed-a645930b9eee None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.321308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4343a-17bf-429b-aaed-a645930b9eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.351025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-f6d4343a-17bf-429b-aaed-a645930b9eee None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:40.352982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f6d4343a-17bf-429b-aaed-a645930b9eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:40.416165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87eec53-54a7-45a3-b1fa-2a57bac36c46 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.441729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 446/895] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:40.454769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.455407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.455682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.455986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.456618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:40.456920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.457821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4343a-17bf-429b-aaed-a645930b9eee None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.458375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.466838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:40.478344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 450/896] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:40.479362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.512787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:40.576504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4343a-17bf-429b-aaed-a645930b9eee None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.613883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95bad7-4af4-4f30-9678-dd360aa35215 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.618005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.618823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.619245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.619680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.620433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.621979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:40.631912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:40.632850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:40.633762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:40.641077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 447/897] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:40.642392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.655044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.655680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.656405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.656734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.666356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.668399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:40.685160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:40.686169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:40.687107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:40.689349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 451/898] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:40.692082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.756811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:40.778333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc28e-ac69-4bc0-9f2c-43fac94c8c61 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.790835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:40.806824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03ed08-68f2-46c7-817d-0c16f5abb7c5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.824392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce263f6b-9ec3-4dd5-adbf-33db5036fbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.825560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce263f6b-9ec3-4dd5-adbf-33db5036fbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.826343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce263f6b-9ec3-4dd5-adbf-33db5036fbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.826814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce263f6b-9ec3-4dd5-adbf-33db5036fbf6 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.827634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce263f6b-9ec3-4dd5-adbf-33db5036fbf6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:40.828151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce263f6b-9ec3-4dd5-adbf-33db5036fbf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.830511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce263f6b-9ec3-4dd5-adbf-33db5036fbf6 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.837515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 448/899] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:40.842718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70267-08b2-4c22-b2f6-b040f81d29a9 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.843008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70267-08b2-4c22-b2f6-b040f81d29a9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.843504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70267-08b2-4c22-b2f6-b040f81d29a9 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.844669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 449/900] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:40.845965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f70267-08b2-4c22-b2f6-b040f81d29a9 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:40.850120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25338e-4dfd-482f-a4e8-303818cb24ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:40.850732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25338e-4dfd-482f-a4e8-303818cb24ee None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.851014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25338e-4dfd-482f-a4e8-303818cb24ee None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.851289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25338e-4dfd-482f-a4e8-303818cb24ee None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.852024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25338e-4dfd-482f-a4e8-303818cb24ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:40.852333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25338e-4dfd-482f-a4e8-303818cb24ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:40.854055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25338e-4dfd-482f-a4e8-303818cb24ee None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:40.873443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 452/901] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:40.873871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2377f83-08d2-4ccd-a5af-f912ba530cc0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:40.874159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2377f83-08d2-4ccd-a5af-f912ba530cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:40.874440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2377f83-08d2-4ccd-a5af-f912ba530cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:40.892621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d2377f83-08d2-4ccd-a5af-f912ba530cc0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:40.894176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d2377f83-08d2-4ccd-a5af-f912ba530cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:40.963332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 453/902] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:40.963505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f70267-08b2-4c22-b2f6-b040f81d29a9 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:40.964542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2377f83-08d2-4ccd-a5af-f912ba530cc0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:41.015683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:41.018379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.019030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.019388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.019791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.020432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:41.020885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.022309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.027966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 450/903] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:41.028609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:41.066204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:41.081659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2377f83-08d2-4ccd-a5af-f912ba530cc0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:41.119291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.120140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.120416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.121055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.135829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56273e6e-947a-439b-9977-1e4458e5c965 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:41.171239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.172200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.172478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.172790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.173419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.174778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.191744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 451/904] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:41 2022] POST /identity/v3/services => generated 241 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:41.192727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:41.215201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.217840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.287822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:41.312066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:41.333161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d21771-76cb-4768-a642-9542a85c60a5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:41.370560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 454/905] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:40 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:19:41.372092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:41.376218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.376838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.377117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.377426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.378146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] RBAC: Adding query filter params (interface=public, service_id=88b95ccc2b0c41e2b2f12b0c7a903b7a, region_id=RegionOne) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:41.378459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.379836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.387781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 452/906] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:19:41 2022] GET /identity/v3/endpoints?service_id=88b95ccc2b0c41e2b2f12b0c7a903b7a&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:41.392930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:41.492631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8eee6-08cb-4c15-8c52-d840a49572ec None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:41.517671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc39b38-db5e-4e5d-a14e-006992233e0a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:41.533979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.535007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.535811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.536239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.549504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.551731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.555630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:41.559480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.560135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.560424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.560706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.561841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.563858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:41.571977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:41.572930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:41.573875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:41.573875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:41.573875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:41.576083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 455/908] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:41.577354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 453/908] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:41 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:19:41.587066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:41.699326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36aa7f-4256-408c-b601-5c264e64b0a0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:41.743736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.744353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.744639 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.744915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.745648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:41.745957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.747438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.761120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 456/909] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:41.766969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:41.792101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:41.878894 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0a5d-6be9-418a-bc28-1fbc054ca549 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:41.913939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:41.914569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:41.914844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:41.915111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:41.922055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:41.923545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:41.935049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:41.936069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:41.937375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 454/910] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:41.942341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:42.006400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:42.052794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21e05a-6935-4b29-bbec-25618b65dfac None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:42.088234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9bfa9-881f-4fc0-8a72-063c095ab369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:42.088845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9bfa9-881f-4fc0-8a72-063c095ab369 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:42.089126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9bfa9-881f-4fc0-8a72-063c095ab369 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:42.089437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9bfa9-881f-4fc0-8a72-063c095ab369 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:42.090130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9bfa9-881f-4fc0-8a72-063c095ab369 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:42.090434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9bfa9-881f-4fc0-8a72-063c095ab369 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:42.091871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9bfa9-881f-4fc0-8a72-063c095ab369 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:42.105027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 457/911] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:42.110624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:42.193013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:42.213808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6177ff-1d50-4ba8-bfc5-015f4d4b8602 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:42.248428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e442a1a-66cd-48a9-8853-242c3390a2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:42.249040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e442a1a-66cd-48a9-8853-242c3390a2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:42.249396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e442a1a-66cd-48a9-8853-242c3390a2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:42.249669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e442a1a-66cd-48a9-8853-242c3390a2d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:42.250444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e442a1a-66cd-48a9-8853-242c3390a2d8 None admin] RBAC: Adding query filter params (role.id=aff6df9c4ba4488ea98690de7d647806, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=9c8a6dd668474aec9274ca29756be19d) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:42.250754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e442a1a-66cd-48a9-8853-242c3390a2d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:42.252197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e442a1a-66cd-48a9-8853-242c3390a2d8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:42.258676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 455/912] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:42 2022] GET /identity/v3/role_assignments?user.id=9c8a6dd668474aec9274ca29756be19d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=aff6df9c4ba4488ea98690de7d647806 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:42.400444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf0b07-8e27-4bc9-b752-6e599bcbe357 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:42.400851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf0b07-8e27-4bc9-b752-6e599bcbe357 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:42.401027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf0b07-8e27-4bc9-b752-6e599bcbe357 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:42.402748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 458/913] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:42.408601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55527105-bbbf-4a29-9802-c8104d2e29fa None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:42.408898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55527105-bbbf-4a29-9802-c8104d2e29fa None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:42.409160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55527105-bbbf-4a29-9802-c8104d2e29fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:42.429942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-55527105-bbbf-4a29-9802-c8104d2e29fa None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:42.431174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-55527105-bbbf-4a29-9802-c8104d2e29fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:42.505388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 456/914] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:42.525252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf0b07-8e27-4bc9-b752-6e599bcbe357 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:42.633721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf0b07-8e27-4bc9-b752-6e599bcbe357 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:42.654702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:42.673851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:42.674479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:42.674757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:42.675028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:42.675671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:42.677045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:42.686043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:42.687197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:42.688212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 459/915] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:42.693037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55527105-bbbf-4a29-9802-c8104d2e29fa None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:42.770551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:42.800611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55527105-bbbf-4a29-9802-c8104d2e29fa None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:42.837632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:42.838246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:42.838525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:42.838798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:42.839431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:42.839734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:42.841122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:42.851481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 457/916] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:42.853169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:42.962184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f384f1-420f-4543-bcfc-9f3d8c24c912 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:43.001747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2866-4741-413e-a342-29cdbf31b901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:43.002437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2866-4741-413e-a342-29cdbf31b901 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.002716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2866-4741-413e-a342-29cdbf31b901 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.002984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2866-4741-413e-a342-29cdbf31b901 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.003608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f2866-4741-413e-a342-29cdbf31b901 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:43.003909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f2866-4741-413e-a342-29cdbf31b901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:43.005290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f2866-4741-413e-a342-29cdbf31b901 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:43.011359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 460/917] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 10 12:19:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:43.016988 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:43.123517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868216c-5004-431b-bc9a-8f7bec7467d5 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:43.159801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b4c7a-9275-415e-8149-3b39e8db3604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:43.160711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b4c7a-9275-415e-8149-3b39e8db3604 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.161193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b4c7a-9275-415e-8149-3b39e8db3604 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.161693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b4c7a-9275-415e-8149-3b39e8db3604 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.162622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b4c7a-9275-415e-8149-3b39e8db3604 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:43.164195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b4c7a-9275-415e-8149-3b39e8db3604 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:43.172702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:43.193857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 458/918] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:19:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:43.376908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab0c1-563d-4f77-bdae-8d9651c2d4fd None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.377296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab0c1-563d-4f77-bdae-8d9651c2d4fd None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.377476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab0c1-563d-4f77-bdae-8d9651c2d4fd None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.378941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 461/919] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:43.384873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e222a3b-3c4d-48c1-ba3b-8e8e8e55b947 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.385162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e222a3b-3c4d-48c1-ba3b-8e8e8e55b947 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.385601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e222a3b-3c4d-48c1-ba3b-8e8e8e55b947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.404657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-0e222a3b-3c4d-48c1-ba3b-8e8e8e55b947 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:43.405892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0e222a3b-3c4d-48c1-ba3b-8e8e8e55b947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:43.492648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 459/920] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:43.509808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ab0c1-563d-4f77-bdae-8d9651c2d4fd None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:43.562157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:43.621297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ab0c1-563d-4f77-bdae-8d9651c2d4fd None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:43.656146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92807f-9089-4d97-b26d-c6afb41b8e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:43.656694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92807f-9089-4d97-b26d-c6afb41b8e48 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.656972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92807f-9089-4d97-b26d-c6afb41b8e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.657243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92807f-9089-4d97-b26d-c6afb41b8e48 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.657930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92807f-9089-4d97-b26d-c6afb41b8e48 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:43.659337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92807f-9089-4d97-b26d-c6afb41b8e48 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-ba92807f-9089-4d97-b26d-c6afb41b8e48 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:43.668175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:43.669147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:43.670010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 462/921] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:43.675367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e222a3b-3c4d-48c1-ba3b-8e8e8e55b947 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:43.781729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e222a3b-3c4d-48c1-ba3b-8e8e8e55b947 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:43.804973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:43.822179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:43.823040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.823525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.824767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.825625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:43.826211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:43.827840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:43.833342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381875e-d197-42ec-81d1-9f0eaaaddf3c None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.833929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381875e-d197-42ec-81d1-9f0eaaaddf3c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.834422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381875e-d197-42ec-81d1-9f0eaaaddf3c None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.836224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 463/923] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:43.836329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 460/923] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:43.840170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:43.841549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177286-8fac-49c6-b94d-11e1b7cc173b None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.842037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177286-8fac-49c6-b94d-11e1b7cc173b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.842525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177286-8fac-49c6-b94d-11e1b7cc173b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.861824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f5177286-8fac-49c6-b94d-11e1b7cc173b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:43.863802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f5177286-8fac-49c6-b94d-11e1b7cc173b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:43.941295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 464/924] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:43.948607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6722d3f-39b2-4cab-a39e-5898fb6b5ecf None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:43.958344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177286-8fac-49c6-b94d-11e1b7cc173b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:43.986585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:43.987199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:43.987480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:43.987756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:43.988502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:43.988813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:43.990875 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.001069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 461/925] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:44.002415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.003972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:44.058629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:44.067738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177286-8fac-49c6-b94d-11e1b7cc173b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.103913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.104562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.104906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.105243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.105958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:44.107489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:44.117501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:44.118450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 12:19:44.119643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:44.121568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88239d27-d1ca-41cd-92f3-b6e3ee31530d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.123847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 465/926] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 10 12:19:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:44.125383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.160439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.161482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.161992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.162445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.163327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] RBAC: Adding query filter params (interface=public, service_id=880868d071204a87a9b326072ee4ef64, region_id=RegionOne) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:44.163817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:44.165391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.175692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 462/927] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:19:44 2022] GET /identity/v3/endpoints?service_id=880868d071204a87a9b326072ee4ef64&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:44.177230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.230499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8816-ac46-444b-bdae-b2d2934b6bea None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.232054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:44.268191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.268838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.269178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.269544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.270244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:44.270615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:44.272385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.281793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 466/928] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:44.282843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.293919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd27cb-37da-4ee9-b924-a939661f49c8 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.310245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:44.335667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ec9a6-5cfc-4fb2-8abf-47900e425263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.336287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ec9a6-5cfc-4fb2-8abf-47900e425263 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.336567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ec9a6-5cfc-4fb2-8abf-47900e425263 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.336842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ec9a6-5cfc-4fb2-8abf-47900e425263 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.337498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ec9a6-5cfc-4fb2-8abf-47900e425263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:44.338858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ec9a6-5cfc-4fb2-8abf-47900e425263 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.345015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 463/929] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:44 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:19:44.351739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea1acf-751b-489f-85f3-80e8517e6823 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.352830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea1acf-751b-489f-85f3-80e8517e6823 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.353291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea1acf-751b-489f-85f3-80e8517e6823 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.360813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 464/930] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:44.363352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3617e-ad39-41e0-906f-2023d8ee150d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.364215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3617e-ad39-41e0-906f-2023d8ee150d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.364643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3617e-ad39-41e0-906f-2023d8ee150d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.390532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-41b3617e-ad39-41e0-906f-2023d8ee150d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:44.391748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-41b3617e-ad39-41e0-906f-2023d8ee150d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:44.412433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc557dea-b3e3-4950-96ff-3c0da25bfe16 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.465630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.466775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.467260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.467733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.473409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 465/931] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:44.473553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:44.473553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:44.473553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.478840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 467/932] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:44.484871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3617e-ad39-41e0-906f-2023d8ee150d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.494506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.538598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:44.613595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3617e-ad39-41e0-906f-2023d8ee150d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.622879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:44.638420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e99be0-a931-4fa4-8ae9-710ba8fe237d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.652500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.653342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.653758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.654219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.655023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:44.656567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.685501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 466/933] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:19:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:44.686030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.686935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.687289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.687652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.700964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:44.706114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:44.707299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:44.708325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:44.710462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 468/934] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 10 12:19:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:44.715569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.831277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbef59-3983-4ded-a554-76193e22f359 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:44.870914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:44.871533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:44.871813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:44.872123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:44.872808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:44.873168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:44.874602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:44.883506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 467/935] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 10 12:19:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:44.889057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:44.979272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:44.994522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a82d3a-8da6-40e5-9ce0-60a0c39ae1a1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.029464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.030099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.030377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.030651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.042431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.043901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:45.058051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:45.059034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:45.060527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:45.060527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:45.060527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:45.060527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 469/936] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:45.065194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.094813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:45.175203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141a712-ad3c-4dc5-880e-936133d660b6 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.210865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873107c2-0516-475c-af15-bff07da3a049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.211833 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873107c2-0516-475c-af15-bff07da3a049 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.212342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873107c2-0516-475c-af15-bff07da3a049 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.212814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873107c2-0516-475c-af15-bff07da3a049 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.213709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873107c2-0516-475c-af15-bff07da3a049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:45.214217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873107c2-0516-475c-af15-bff07da3a049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.215803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873107c2-0516-475c-af15-bff07da3a049 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.228905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 468/937] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:45.234607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.276859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6178e-a623-4f3d-95a0-5407177e6a33 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.277202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6178e-a623-4f3d-95a0-5407177e6a33 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.277458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6178e-a623-4f3d-95a0-5407177e6a33 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.279060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 469/938] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:45.284046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9e22-d12d-4766-815f-ec04778bd483 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.284332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9e22-d12d-4766-815f-ec04778bd483 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.284604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9e22-d12d-4766-815f-ec04778bd483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.302705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fbee9e22-d12d-4766-815f-ec04778bd483 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:45.303887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fbee9e22-d12d-4766-815f-ec04778bd483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:45.361363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439df07-d342-4a27-a05c-9fc9c8e57c5c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.377572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 470/939] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:45.394530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee9e22-d12d-4766-815f-ec04778bd483 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.402404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.403327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.404389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.411627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:45.418897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.421621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.424652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:45.439622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:45.440620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:45.441604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:45.444267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 470/940] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:19:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:45.450027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.509538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee9e22-d12d-4766-815f-ec04778bd483 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.547086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.548077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.548573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.549043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.549914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.551549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.555018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING keystone.server.flask.common [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 12:19:45.572123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bbbfd-1a44-4308-9dd8-b2a000a9f8ef None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.582098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.sql.core [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 12:19:45.582098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 12:19:45.582098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [parameters: {'id': '7b2b5dc583934c50927a2f5d1d360a0f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 12:19:45.582098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80063) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b2b5dc583934c50927a2f5d1d360a0f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b2b5dc583934c50927a2f5d1d360a0f. Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:19:45.605540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application result.read() Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 12:19:45.606369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application yield resource Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise self.value Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application session.commit() Aug 10 12:19:45.607298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self.session.flush() Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application compat.raise_( Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 12:19:45.608209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 12:19:45.609111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application result.read() Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:19:45.616745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [parameters: {'id': '7b2b5dc583934c50927a2f5d1d360a0f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.619276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:45.621424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b2b5dc583934c50927a2f5d1d360a0f. Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 471/941] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 10 12:19:45 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:45.623759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.625214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.625214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.625214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.625214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.626062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:45.626062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.626334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.650298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 471/942] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 10 12:19:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:45.658121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.705205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:45.714035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:45.742542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce5b1-8f19-4727-88ee-ac6afd6d1767 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.762337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70568a3-fa6f-48fc-ac6c-c461ff1f4242 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.779085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.779915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.780346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.780693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.788589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.790563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.798932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.799636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.799987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.800329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:45.803614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:45.804563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:45.805478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:45.810214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 472/943] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:45.811225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.811412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.812705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:45.827360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:45.829304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:45.831293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:45.831293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:45.831293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:45.833334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 472/944] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:45.834741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:45.935020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefa0fe-34d5-445c-903d-12ad2d299237 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.954210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772dafb-1b01-40bc-8cc5-a032871bcde8 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:45.965827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:45.968436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:45.972166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.973253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.973771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:45.975375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:45.976413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:45.976819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:45.978847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:45.998688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:45.999542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:45.999977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 473/945] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:46.000178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.000706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.001534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:46.001969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:46.003489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:46.018499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 473/946] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:46.027475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:46.140291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8b211-8701-49ec-9264-f8356bc5934f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:46.175682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:46.176459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.176853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.177221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.184161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:46.185688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:46.197203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:46.198109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:46.199007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 474/947] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:46.204301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:46.316634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba7774-b361-48b6-ba00-28e63c7c4571 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:46.354016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e49080-908c-4fdc-8ae7-c1fd63353acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:46.354621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e49080-908c-4fdc-8ae7-c1fd63353acb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.354895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e49080-908c-4fdc-8ae7-c1fd63353acb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.355163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e49080-908c-4fdc-8ae7-c1fd63353acb None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.355851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e49080-908c-4fdc-8ae7-c1fd63353acb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:46.356154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e49080-908c-4fdc-8ae7-c1fd63353acb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:46.357640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e49080-908c-4fdc-8ae7-c1fd63353acb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:46.370511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 474/948] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:46.376298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:46.430611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:46.482918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4d9b2-5fa7-45e4-ab44-fce463b17c94 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:46.488880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251b3c-0a76-41b1-82eb-3041f797c4f7 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.489168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251b3c-0a76-41b1-82eb-3041f797c4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.489485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251b3c-0a76-41b1-82eb-3041f797c4f7 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.491614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 475/949] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:46.496538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6877a-4620-4e01-b347-c02f4a71d0e8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.496826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6877a-4620-4e01-b347-c02f4a71d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.497106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6877a-4620-4e01-b347-c02f4a71d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.515874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f6f6877a-4620-4e01-b347-c02f4a71d0e8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:46.517062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f6f6877a-4620-4e01-b347-c02f4a71d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:46.518912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:46.519573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.519856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.520124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/9c8a6dd668474aec9274ca29756be19d/roles/aff6df9c4ba4488ea98690de7d647806` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.522397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=9c8a6dd668474aec9274ca29756be19d, role_id=aff6df9c4ba4488ea98690de7d647806)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:46.523813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:46.539100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 475/950] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:19:46 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/9c8a6dd668474aec9274ca29756be19d/roles/aff6df9c4ba4488ea98690de7d647806 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:19:46.543810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:46.604726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 476/951] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:46.628156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:46.739395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1931bdd-7da7-4000-9bd0-e122fe1c2df6 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:46.750825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:46.778325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ee66a-09c9-4a74-811b-44c1db0e55e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:46.779060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ee66a-09c9-4a74-811b-44c1db0e55e9 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.779409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ee66a-09c9-4a74-811b-44c1db0e55e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.779749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ee66a-09c9-4a74-811b-44c1db0e55e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.780408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ee66a-09c9-4a74-811b-44c1db0e55e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:46.781933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ee66a-09c9-4a74-811b-44c1db0e55e9 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-4e0ee66a-09c9-4a74-811b-44c1db0e55e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:46.791049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:46.792251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:46.793261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:46.794019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 476/952] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:46.800115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6877a-4620-4e01-b347-c02f4a71d0e8 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:46.884360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bcab7-8093-4d17-84fc-74efc6d81cdd None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.884815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bcab7-8093-4d17-84fc-74efc6d81cdd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.885192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bcab7-8093-4d17-84fc-74efc6d81cdd None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.886926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 477/953] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:46.891876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefdf2d-8d2c-4a69-819e-b1236ace4a80 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.892260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefdf2d-8d2c-4a69-819e-b1236ace4a80 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.892633 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefdf2d-8d2c-4a69-819e-b1236ace4a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.910880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6877a-4620-4e01-b347-c02f4a71d0e8 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:46.912030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-aeefdf2d-8d2c-4a69-819e-b1236ace4a80 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:46.913884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-aeefdf2d-8d2c-4a69-819e-b1236ace4a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:46.946205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:46.946823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:46.947095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:46.947371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:46.948002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:46.948311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:46.949731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:46.955318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 477/954] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:46.961258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:46.986280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 478/955] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:46.987792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:47.007910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefdf2d-8d2c-4a69-819e-b1236ace4a80 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:47.067724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0ad0-f9f7-4efa-b344-b42b07b3a766 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:47.107043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:47.108049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:47.108760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:47.109216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:47.110672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:47.111161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:47.112467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefdf2d-8d2c-4a69-819e-b1236ace4a80 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:47.112926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:47.116548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:47.127489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 478/956] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:47.133306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:47.158661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:47.159288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:47.159564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:47.159832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:47.160413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:47.161932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:47.170815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:47.172022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:47.172911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 479/957] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:47.178544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:47.246252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:47.268947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5444-dfba-4951-a51d-b3267d686259 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:47.292957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590e53-4da5-43fe-9850-2fa5661ed9e4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:47.319149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bb6a3-c7a8-47f4-9dc5-38639540dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:47.320633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bb6a3-c7a8-47f4-9dc5-38639540dfb7 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:47.321200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bb6a3-c7a8-47f4-9dc5-38639540dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:47.321702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bb6a3-c7a8-47f4-9dc5-38639540dfb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:47.323202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bb6a3-c7a8-47f4-9dc5-38639540dfb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:47.325032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bb6a3-c7a8-47f4-9dc5-38639540dfb7 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:47.332493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:47.333373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:47.333746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:47.334147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:47.334825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:47.335236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:47.336736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:47.342573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 480/958] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:47.348567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:47.354564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 479/959] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:19:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:19:47.388107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:47.469959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe3e83-6897-4cbb-a6e3-1a308df4f88b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:47.519923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7b509-7503-4527-88e3-a39b6438b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:47.520891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7b509-7503-4527-88e3-a39b6438b193 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:47.521471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7b509-7503-4527-88e3-a39b6438b193 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:47.521948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7b509-7503-4527-88e3-a39b6438b193 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:47.522818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7b509-7503-4527-88e3-a39b6438b193 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:47.523331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7b509-7503-4527-88e3-a39b6438b193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:47.524910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7b509-7503-4527-88e3-a39b6438b193 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:47.530786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 481/960] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:48.204400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f88d0-2728-40da-87fa-df318324884d None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:48.204925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f88d0-2728-40da-87fa-df318324884d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:48.204925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f88d0-2728-40da-87fa-df318324884d None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:48.206581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 480/961] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:48.212058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c33b1d-6bae-4bec-b1bd-867f326fdcc5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:48.212476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c33b1d-6bae-4bec-b1bd-867f326fdcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:48.212654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c33b1d-6bae-4bec-b1bd-867f326fdcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:48.230794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-09c33b1d-6bae-4bec-b1bd-867f326fdcc5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:48.231981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-09c33b1d-6bae-4bec-b1bd-867f326fdcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:48.305387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 482/962] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:48.326262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f88d0-2728-40da-87fa-df318324884d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:48.436095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f88d0-2728-40da-87fa-df318324884d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:48.471983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:48.472930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:48.473446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:48.473942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:48.475308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:48.476946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:48.480014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING keystone.server.flask.common [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 12:19:48.493822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.sql.core [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 12:19:48.493822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 12:19:48.493822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [parameters: {'user_id': '484576939db34e49ab1b0d2940cb68ed', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 12:19:48.493822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80064) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result.read() Aug 10 12:19:48.501045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:19:48.501930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application yield resource Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application next(self.gen) Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise self.value Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application session.commit() Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 12:19:48.502975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.session.flush() Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application compat.raise_( Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 12:19:48.503891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 12:19:48.504808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application conn.query(q) Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application result.read() Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 12:19:48.505734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [parameters: {'user_id': '484576939db34e49ab1b0d2940cb68ed', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:48.508428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 12:19:48.510365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 12:19:48.512505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 12:19:48.512505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 12:19:48.512505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 12:19:48.512505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 12:19:48.512505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 12:19:48.512505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:48.515012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 481/963] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 10 12:19:48 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:48.521492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c33b1d-6bae-4bec-b1bd-867f326fdcc5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:48.557651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:48.636487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c33b1d-6bae-4bec-b1bd-867f326fdcc5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:48.678031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925cc4c-c751-42dd-81ce-e2f66a01c170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:48.678660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925cc4c-c751-42dd-81ce-e2f66a01c170 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:48.678994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925cc4c-c751-42dd-81ce-e2f66a01c170 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:48.679269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925cc4c-c751-42dd-81ce-e2f66a01c170 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:48.686768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925cc4c-c751-42dd-81ce-e2f66a01c170 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:48.688221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925cc4c-c751-42dd-81ce-e2f66a01c170 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-8925cc4c-c751-42dd-81ce-e2f66a01c170 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:48.699768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:48.700705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:48.701811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 483/964] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:19:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:48.710662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:48.716534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:48.739188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bbe33-93cb-4391-94c4-75a137310a6f None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:48.739659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bbe33-93cb-4391-94c4-75a137310a6f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:48.740121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bbe33-93cb-4391-94c4-75a137310a6f None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:48.741879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 484/965] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:48.746618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8bb8f-d1f9-4c28-af91-10f1a88db0c1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:48.747047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8bb8f-d1f9-4c28-af91-10f1a88db0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:48.747482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8bb8f-d1f9-4c28-af91-10f1a88db0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:48.766128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-48c8bb8f-d1f9-4c28-af91-10f1a88db0c1 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:48.767862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-48c8bb8f-d1f9-4c28-af91-10f1a88db0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:48.829191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02dd3d0-345c-417d-ad54-b4403066fb23 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:48.867864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 485/966] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:48.882532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:48.883141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:48.883417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:48.883683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:48.884374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:48.884671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:48.886388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:48.891436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8bb8f-d1f9-4c28-af91-10f1a88db0c1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:48.907716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:48.931369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 482/967] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 10 12:19:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:49.008804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8bb8f-d1f9-4c28-af91-10f1a88db0c1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.047837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.048881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.049404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.049890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.062357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:49.067515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:49.069189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:49.070147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:49.073162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 486/968] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 10 12:19:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:49.079440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.152465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:49.192907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20786-a30b-4d1d-b354-bbf8a2b8df60 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.230365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.231001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.231247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.231520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.232149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:49.232450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:49.233840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:49.242329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 483/969] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 10 12:19:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:49.248724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.322218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:49.354193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1510b7-7cf0-495f-88e4-770a288aa8d3 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.389229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfaed0-f0e0-4cc7-9ed2-66dc6d0f0b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.390029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfaed0-f0e0-4cc7-9ed2-66dc6d0f0b5a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.390393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfaed0-f0e0-4cc7-9ed2-66dc6d0f0b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.390786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfaed0-f0e0-4cc7-9ed2-66dc6d0f0b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.401112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfaed0-f0e0-4cc7-9ed2-66dc6d0f0b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:49.402687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfaed0-f0e0-4cc7-9ed2-66dc6d0f0b5a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-95bfaed0-f0e0-4cc7-9ed2-66dc6d0f0b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:49.417233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:49.418412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:49.419504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:49.419504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:49.419504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:49.425060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 487/970] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:49.426285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.471379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dd157-6d92-44e8-8508-450592bc1395 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.471888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dd157-6d92-44e8-8508-450592bc1395 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.472399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dd157-6d92-44e8-8508-450592bc1395 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.475131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 488/971] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:49.480546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f1bb-faaa-4057-8ae2-1fa4411651b8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.480829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f1bb-faaa-4057-8ae2-1fa4411651b8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.481103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f1bb-faaa-4057-8ae2-1fa4411651b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.499688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5306f1bb-faaa-4057-8ae2-1fa4411651b8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:49.500865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5306f1bb-faaa-4057-8ae2-1fa4411651b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:49.535118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd730a-9b49-4a53-b91f-42813a5a423a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.558360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:49.570367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.570961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.571235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.571503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.572134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:49.572284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 489/972] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:49.572439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:49.575862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d0096-0b51-4b86-b231-8576f0ca4da6 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.575862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d0096-0b51-4b86-b231-8576f0ca4da6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.576070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:49.576143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d0096-0b51-4b86-b231-8576f0ca4da6 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.576143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 490/973] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:49.580581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f4211-38ae-4106-a676-cb42badb7713 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.580852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f4211-38ae-4106-a676-cb42badb7713 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.581117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f4211-38ae-4106-a676-cb42badb7713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.591928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 484/974] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:49.597212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.599419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-4f6f4211-38ae-4106-a676-cb42badb7713 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:49.600602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4f6f4211-38ae-4106-a676-cb42badb7713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:49.649638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:49.666521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 491/975] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:49.667420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f4211-38ae-4106-a676-cb42badb7713 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.710159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114b87b-8d16-4d8c-b200-07101ce16c2d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.743656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.744285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.744562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.744833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.745440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:49.746804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:49.755266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:49.761647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:49.762772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 485/976] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:49.763748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.765851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f4211-38ae-4106-a676-cb42badb7713 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.799149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.799772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.800051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.800322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.807737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:49.809176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:49.811119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:49.820829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:49.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 492/977] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:19:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:49.823164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.866156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57722560-07de-42b3-a771-5cde7d729bd8 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.871157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:49.899824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.900425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.900704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.900981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.901649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:49.903020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:49.918508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 486/978] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:49 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:49.919302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:49.941181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f336f8-6de4-4fbe-99db-bee77b6b2a11 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:49.988277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:49.988905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:49.989188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:49.989480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:49.990769 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:49.991014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:49.992410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.002753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 493/979] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:50.004012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:50.042241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87b84b-41ec-4da9-bbca-a1efa7750e2a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.052296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:50.086513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.087038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.087941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.087941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.088298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:50.088595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.090057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.109677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 487/980] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 10 12:19:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:50.115564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:50.123514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f51fbe-c059-4aaa-802e-53359fd9699e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.127038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:50.160247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.160939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.161302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.161652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.163012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:50.163321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.164693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.169822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 494/981] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:50.175826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:50.226963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda028a6-6265-480e-9cbd-c397d4f31d4a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.261222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.261984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.262096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.262365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.283097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592fade-fea3-466e-93d6-bf889c3bcf40 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.300534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:50.321700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.322316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.322621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.322902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.323619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] RBAC: Adding query filter params (interface=admin, service_id=880868d071204a87a9b326072ee4ef64, region_id=RegionOne) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:50.323930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.325318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.336071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 495/982] 10.222.0.65 () {56 vars in 1249 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/endpoints?service_id=880868d071204a87a9b326072ee4ef64&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:50.337230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:50.338583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.340045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.374654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:50.446500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344ca35-633e-4274-8297-84538f1ad583 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.451408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 488/983] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:19:50.452058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:50.486643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.487344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.487533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.487805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.488415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.489821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.501819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 496/984] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:19:50.523942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:50.563014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4d7d9-9c72-4af5-b865-6789803e92c3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.607575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.608229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.608504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.608773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.620408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.621962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:50.639193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:50.640209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:50.645858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:50.645858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:50.645858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:50.645858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 489/985] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:50.646085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:50.747597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:50.754810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455cd64-ae83-4104-8397-75826a8a483f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.790084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.790692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.791248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.791521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.792159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:50.792457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.793843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.805785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 497/986] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:50.811269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:50.913500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751583-5ed7-4a8a-a329-aa86bbc1cf90 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:50.930801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:50.948859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:50.949514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:50.949795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:50.950063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:50.961588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:50.963057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:50.974322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:50.975237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:50.976120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 490/987] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:50.982946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:51.085944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189f0c4-f370-4eae-b50d-2daa875cfa83 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:51.120133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb456830-9cab-4008-9aa9-2e797e55f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:51.120744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb456830-9cab-4008-9aa9-2e797e55f530 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:51.121024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb456830-9cab-4008-9aa9-2e797e55f530 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:51.121325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb456830-9cab-4008-9aa9-2e797e55f530 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:51.122020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb456830-9cab-4008-9aa9-2e797e55f530 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:51.122324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb456830-9cab-4008-9aa9-2e797e55f530 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:51.123819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb456830-9cab-4008-9aa9-2e797e55f530 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:51.136251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 498/988] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:51.142263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:51.243660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c19f-5e4f-4b18-8afb-c8c1512e3364 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:51.277317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c63d86-4252-4f3a-9624-fde16745fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:51.277986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c63d86-4252-4f3a-9624-fde16745fc64 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:51.278251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c63d86-4252-4f3a-9624-fde16745fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:51.278508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c63d86-4252-4f3a-9624-fde16745fc64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:51.279281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c63d86-4252-4f3a-9624-fde16745fc64 None admin] RBAC: Adding query filter params (role.id=aff6df9c4ba4488ea98690de7d647806, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=9c8a6dd668474aec9274ca29756be19d) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:51.279592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c63d86-4252-4f3a-9624-fde16745fc64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:51.281022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c63d86-4252-4f3a-9624-fde16745fc64 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:51.287611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 491/989] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:19:51 2022] GET /identity/v3/role_assignments?user.id=9c8a6dd668474aec9274ca29756be19d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=aff6df9c4ba4488ea98690de7d647806 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:51.951880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43299788-1afc-43c2-aaf2-3591078bf209 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:51.952400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43299788-1afc-43c2-aaf2-3591078bf209 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:51.952400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43299788-1afc-43c2-aaf2-3591078bf209 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:51.953974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 499/990] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:51.959472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188cba-1777-4214-8983-d3bee5dd9dbf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:51.959831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188cba-1777-4214-8983-d3bee5dd9dbf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:51.960184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e188cba-1777-4214-8983-d3bee5dd9dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:51.979147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1e188cba-1777-4214-8983-d3bee5dd9dbf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:51.980678 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1e188cba-1777-4214-8983-d3bee5dd9dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:52.018433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:52.052011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 492/991] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:52.074409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43299788-1afc-43c2-aaf2-3591078bf209 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:52.093552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:52.178630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43299788-1afc-43c2-aaf2-3591078bf209 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:52.212538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:52.213097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.213415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.213696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.214282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:52.215652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:52.224256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:52.225215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:52.226358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:52.229678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 500/992] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:52.231762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188cba-1777-4214-8983-d3bee5dd9dbf None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:52.334371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e188cba-1777-4214-8983-d3bee5dd9dbf None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:52.370366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ee462-1f41-4cbe-b386-28aa392cacb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:52.370976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ee462-1f41-4cbe-b386-28aa392cacb8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.371254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ee462-1f41-4cbe-b386-28aa392cacb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.371527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ee462-1f41-4cbe-b386-28aa392cacb8 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.372155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ee462-1f41-4cbe-b386-28aa392cacb8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:52.372457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ee462-1f41-4cbe-b386-28aa392cacb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:52.373962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ee462-1f41-4cbe-b386-28aa392cacb8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:52.384148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 493/993] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:52.385670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:52.470547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:52.491198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef1c4c-326a-4a0b-9202-ba86b0196310 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.491506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef1c4c-326a-4a0b-9202-ba86b0196310 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.491749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef1c4c-326a-4a0b-9202-ba86b0196310 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.492383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d6d3d-b7e3-41ae-8c5e-d4911a33519d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:52.496685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 494/994] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:52.498404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a51f7-1ac1-4528-a2a6-d1cb80934765 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.498691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a51f7-1ac1-4528-a2a6-d1cb80934765 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.498968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a51f7-1ac1-4528-a2a6-d1cb80934765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.518623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-182a51f7-1ac1-4528-a2a6-d1cb80934765 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:52.519813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-182a51f7-1ac1-4528-a2a6-d1cb80934765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:52.534902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:52.535512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.535794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.536065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.536691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:52.536999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:52.538388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:52.543457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 501/995] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:52.549057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:52.555876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:52.603329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 495/996] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:52.625992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a51f7-1ac1-4528-a2a6-d1cb80934765 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:52.661204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc83926-a7a2-4987-827d-b793e4076d3a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:52.695403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:52.696005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.696286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.696557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.697250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] RBAC: Adding query filter params (interface=public, service_id=4aaecbfafaf54b648c63db282fe2f4ef, region_id=RegionOne) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:52.697582 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:52.698964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:52.704002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 502/997] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/endpoints?service_id=4aaecbfafaf54b648c63db282fe2f4ef&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:52.709390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:52.719734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:52.733157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a51f7-1ac1-4528-a2a6-d1cb80934765 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:52.770398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:52.771059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.771337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.771611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.772193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:52.773708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:52.782778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:52.783949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:52.784785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 496/998] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:52.790656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:52.818116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40e979-0862-48b6-a297-e7fc16b28501 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:52.822887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:52.856486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:52.857100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.857431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.857700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.858306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:52.859692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:52.867051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 503/999] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:19:52.917283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560dfd9-ada2-4590-bd0f-19a405005fc2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:52.954617 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:52.955233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:52.955512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:52.955851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:52.956492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:52.956822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:52.958243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:52.968252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 497/1000] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:52.969776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:53.049490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:53.080872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8b27-5dee-4b36-88de-13ab9b16ef1d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:53.116516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f4783-9bfd-477f-aca7-3138e74e031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:53.117123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f4783-9bfd-477f-aca7-3138e74e031c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:53.117433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f4783-9bfd-477f-aca7-3138e74e031c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:53.117704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f4783-9bfd-477f-aca7-3138e74e031c None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:53.118326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f4783-9bfd-477f-aca7-3138e74e031c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:53.118628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f4783-9bfd-477f-aca7-3138e74e031c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:53.119983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f4783-9bfd-477f-aca7-3138e74e031c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:53.125165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 504/1001] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 10 12:19:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:53.130620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:53.224136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:53.232391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207f9db-0509-4c4b-8839-ab1d4ea18e83 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:53.266500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c339cf-b0fc-4101-8923-acb8120d48b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:53.267108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c339cf-b0fc-4101-8923-acb8120d48b3 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:53.267385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c339cf-b0fc-4101-8923-acb8120d48b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:53.267655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c339cf-b0fc-4101-8923-acb8120d48b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:53.268320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c339cf-b0fc-4101-8923-acb8120d48b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:53.269717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c339cf-b0fc-4101-8923-acb8120d48b3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:53.288345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 498/1002] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:19:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:53.315684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28310-cab4-4bdb-bb05-e288db4ae982 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:53.315892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28310-cab4-4bdb-bb05-e288db4ae982 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:53.316170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28310-cab4-4bdb-bb05-e288db4ae982 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:53.317768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 505/1003] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:53.324140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be66d9-fbac-4b75-b1aa-ec7a058b8a09 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:53.324473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be66d9-fbac-4b75-b1aa-ec7a058b8a09 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:53.324698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be66d9-fbac-4b75-b1aa-ec7a058b8a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:53.344667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b7be66d9-fbac-4b75-b1aa-ec7a058b8a09 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:53.345996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b7be66d9-fbac-4b75-b1aa-ec7a058b8a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:53.429305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 499/1004] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:53.449205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28310-cab4-4bdb-bb05-e288db4ae982 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:53.559412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28310-cab4-4bdb-bb05-e288db4ae982 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:53.599914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:53.600556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:53.600832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:53.601100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] PATH_INFO: `/v3/services/image` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:53.601722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:53.603366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:53.606219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:53.613439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:53.614469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:53.615927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 506/1005] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 10 12:19:53 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:53.622884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be66d9-fbac-4b75-b1aa-ec7a058b8a09 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:53.703961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:53.725654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be66d9-fbac-4b75-b1aa-ec7a058b8a09 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:53.759992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab88b3f-5031-4c15-ab38-dee194dc255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:53.760597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab88b3f-5031-4c15-ab38-dee194dc255e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:53.760875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab88b3f-5031-4c15-ab38-dee194dc255e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:53.761186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab88b3f-5031-4c15-ab38-dee194dc255e None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:53.761811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab88b3f-5031-4c15-ab38-dee194dc255e None admin] RBAC: Adding query filter params (name=image) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:53.762115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab88b3f-5031-4c15-ab38-dee194dc255e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:53.763475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab88b3f-5031-4c15-ab38-dee194dc255e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:53.773658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 500/1006] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:53.775017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:53.881376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb20e04-b7cd-4541-a65d-094d90cdac09 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:53.914783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724da1c-02c6-4118-8d24-a10f10289bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:53.915387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724da1c-02c6-4118-8d24-a10f10289bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:53.915661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724da1c-02c6-4118-8d24-a10f10289bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:53.915930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724da1c-02c6-4118-8d24-a10f10289bb3 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:53.916554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724da1c-02c6-4118-8d24-a10f10289bb3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:53.916853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724da1c-02c6-4118-8d24-a10f10289bb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:53.918236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724da1c-02c6-4118-8d24-a10f10289bb3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:53.929617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 507/1007] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:54.888163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1d95-863a-4b49-bdab-24902da09f37 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:54.888673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1d95-863a-4b49-bdab-24902da09f37 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:54.888673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1d95-863a-4b49-bdab-24902da09f37 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:54.890155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 501/1008] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:54.894997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5aabf-fb1b-4e0d-a039-0cf5a57888d8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:54.895371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5aabf-fb1b-4e0d-a039-0cf5a57888d8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:54.895570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5aabf-fb1b-4e0d-a039-0cf5a57888d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:54.913332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-64c5aabf-fb1b-4e0d-a039-0cf5a57888d8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:54.914477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-64c5aabf-fb1b-4e0d-a039-0cf5a57888d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:54.977539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:54.986164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 508/1009] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:55.009576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b1d95-863a-4b49-bdab-24902da09f37 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:55.112295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b1d95-863a-4b49-bdab-24902da09f37 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:55.145908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:55.146494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.146772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.147041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.147614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:55.148977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:55.157492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:55.158443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 12:19:55.159273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:55.161593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 502/1010] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:55.165002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5aabf-fb1b-4e0d-a039-0cf5a57888d8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:55.261966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5aabf-fb1b-4e0d-a039-0cf5a57888d8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:55.295647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99014894-dc85-47bb-9e8f-4bd388c00d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:55.296256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014894-dc85-47bb-9e8f-4bd388c00d83 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.296536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014894-dc85-47bb-9e8f-4bd388c00d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.296806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014894-dc85-47bb-9e8f-4bd388c00d83 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.297471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99014894-dc85-47bb-9e8f-4bd388c00d83 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:55.297776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99014894-dc85-47bb-9e8f-4bd388c00d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:55.299145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99014894-dc85-47bb-9e8f-4bd388c00d83 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:55.308838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 509/1011] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:55.310754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:55.322943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:55.323328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460003-6dc6-483c-8ea2-98db3510f62a None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.323617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460003-6dc6-483c-8ea2-98db3510f62a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.323898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460003-6dc6-483c-8ea2-98db3510f62a None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.325241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 510/1012] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:55.330433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aabbd-244f-4d32-892c-7f82b5e20078 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.330716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aabbd-244f-4d32-892c-7f82b5e20078 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.330992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aabbd-244f-4d32-892c-7f82b5e20078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.350381 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d28aabbd-244f-4d32-892c-7f82b5e20078 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:55.351571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d28aabbd-244f-4d32-892c-7f82b5e20078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:55.410588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:55.426660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce87d1c-1114-4cea-93da-3d6f49757b1d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:55.427329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 511/1013] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:55 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:55.449755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28aabbd-244f-4d32-892c-7f82b5e20078 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:55.462351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:55.462965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.463242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.463514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.464243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:55.464443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:55.465864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:55.471207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 503/1014] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 10 12:19:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:55.477148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:55.557704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28aabbd-244f-4d32-892c-7f82b5e20078 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:55.558121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:55.591891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5ce4f-917f-4da8-81e3-190a23bbb4b2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:55.605589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:55.606234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.606630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.606902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.608093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:55.609600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:55.615669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 512/1015] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:55.617412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:55.631552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71878a56-1304-4f04-98ae-443e895bd41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:55.632161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71878a56-1304-4f04-98ae-443e895bd41b None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.632440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71878a56-1304-4f04-98ae-443e895bd41b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.632706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71878a56-1304-4f04-98ae-443e895bd41b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.633434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71878a56-1304-4f04-98ae-443e895bd41b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:55.634737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71878a56-1304-4f04-98ae-443e895bd41b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:55.656275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 504/1016] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:19:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:19:55.685564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:55.735575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183074f7-4cb1-4d9f-9ce7-0f38322bc98d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:55.780109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537e768-30b3-4dc3-9a74-4f68b6f391f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:55.780731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537e768-30b3-4dc3-9a74-4f68b6f391f1 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.781008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537e768-30b3-4dc3-9a74-4f68b6f391f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.781304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537e768-30b3-4dc3-9a74-4f68b6f391f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.781946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537e768-30b3-4dc3-9a74-4f68b6f391f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:55.783396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537e768-30b3-4dc3-9a74-4f68b6f391f1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:55.818020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 513/1017] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:55 2022] POST /identity/v3/projects => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:19:55.942757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f578a86-de5b-4381-b1af-8e76b34c6ed0 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.943240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f578a86-de5b-4381-b1af-8e76b34c6ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.943331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f578a86-de5b-4381-b1af-8e76b34c6ed0 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.944956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 505/1018] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:55.950895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be72f0-1cd4-4b92-b3e1-04d288f26575 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:55.951399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be72f0-1cd4-4b92-b3e1-04d288f26575 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:55.951497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be72f0-1cd4-4b92-b3e1-04d288f26575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:55.970282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e4be72f0-1cd4-4b92-b3e1-04d288f26575 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:55.971471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e4be72f0-1cd4-4b92-b3e1-04d288f26575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:56.043427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 514/1019] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:55 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:56.069487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f578a86-de5b-4381-b1af-8e76b34c6ed0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:56.174984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f578a86-de5b-4381-b1af-8e76b34c6ed0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:56.201625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:56.212877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccb60c-2bf1-4676-b58f-0135b7c20ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:56.213560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb60c-2bf1-4676-b58f-0135b7c20ae2 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:56.213837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb60c-2bf1-4676-b58f-0135b7c20ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:56.214106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccb60c-2bf1-4676-b58f-0135b7c20ae2 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:56.214756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccb60c-2bf1-4676-b58f-0135b7c20ae2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:56.216119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccb60c-2bf1-4676-b58f-0135b7c20ae2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:56.233312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 506/1020] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:19:56 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:19:57.799958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e917244-5fa0-4606-b492-a8cc00fb5905 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:57.800471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e917244-5fa0-4606-b492-a8cc00fb5905 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:57.800471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e917244-5fa0-4606-b492-a8cc00fb5905 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:57.802016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 515/1021] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:57.807085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647df20-0498-420d-b9ef-28ea8ea891a6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:57.807509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647df20-0498-420d-b9ef-28ea8ea891a6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:57.807765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647df20-0498-420d-b9ef-28ea8ea891a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:57.825754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e647df20-0498-420d-b9ef-28ea8ea891a6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:57.826944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e647df20-0498-420d-b9ef-28ea8ea891a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:57.856008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c0423-ddf1-41c1-8b97-c66cef15aa1a None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:57.856195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c0423-ddf1-41c1-8b97-c66cef15aa1a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:57.856464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c0423-ddf1-41c1-8b97-c66cef15aa1a None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:57.857967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 516/1022] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:19:57.862656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3495fc0-3fad-486d-92fa-49cc7aac27ea None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:57.862937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3495fc0-3fad-486d-92fa-49cc7aac27ea None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:57.863205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3495fc0-3fad-486d-92fa-49cc7aac27ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:57.880468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e3495fc0-3fad-486d-92fa-49cc7aac27ea None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:57.881667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e3495fc0-3fad-486d-92fa-49cc7aac27ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:57.904230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 507/1023] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:19:57.912280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:57.928048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647df20-0498-420d-b9ef-28ea8ea891a6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:57.961415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 517/1024] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:57.984115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3495fc0-3fad-486d-92fa-49cc7aac27ea None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.039031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647df20-0498-420d-b9ef-28ea8ea891a6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.053140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.077438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.078064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.078339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.078619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.089364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:19:58.091808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3495fc0-3fad-486d-92fa-49cc7aac27ea None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:19:58.093998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:58.094923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:58.095865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 508/1025] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 10 12:19:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:58.101010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.130933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.131562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.131844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.132121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.133233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.134722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.136405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 518/1026] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:58.141879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.174294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.215267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666dbf-dbd7-4abb-86fc-d20837ee163d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.248224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7843c-883d-45bd-b406-a70540dd8b3e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.250549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704c48e-3f6a-4d07-9884-67fc2937b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.251147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704c48e-3f6a-4d07-9884-67fc2937b20e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.251423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704c48e-3f6a-4d07-9884-67fc2937b20e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.251690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704c48e-3f6a-4d07-9884-67fc2937b20e None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.252481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704c48e-3f6a-4d07-9884-67fc2937b20e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:58.252797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704c48e-3f6a-4d07-9884-67fc2937b20e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.254216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704c48e-3f6a-4d07-9884-67fc2937b20e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.262896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 509/1027] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:58.264179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526231b-2e1d-4213-9f4d-7f5c0dd2c82c None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.264460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526231b-2e1d-4213-9f4d-7f5c0dd2c82c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.264728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526231b-2e1d-4213-9f4d-7f5c0dd2c82c None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.266073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 510/1028] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:19:58.270903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a4dd4-0085-43fd-a707-9d3b31a4cbb2 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.271764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a4dd4-0085-43fd-a707-9d3b31a4cbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.272325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a4dd4-0085-43fd-a707-9d3b31a4cbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.288242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.288918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.289197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.289515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.290134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.291561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.292383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-743a4dd4-0085-43fd-a707-9d3b31a4cbb2 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:19:58.293597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-743a4dd4-0085-43fd-a707-9d3b31a4cbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:19:58.313252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 519/1029] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:19:58 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:19:58.314088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.334263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.369016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 511/1030] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:19:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:19:58.392405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a4dd4-0085-43fd-a707-9d3b31a4cbb2 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.417729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.434098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50caa7-b4ab-460d-8f7d-12cdfc7085e1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.473651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.474747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.475035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.475312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.488455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.490657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:58.506051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:58.507219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:58.508212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:58.508212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:58.508212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:58.508212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 520/1031] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:58.511104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a4dd4-0085-43fd-a707-9d3b31a4cbb2 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.513037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.556259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.556893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.557161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.557461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] PATH_INFO: `/v3/services/image` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.558048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.559418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:58.568862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:58.569920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:58.570973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 512/1032] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:58.579240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.616031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.632895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e040d-45e5-4b81-8b07-27f32113d593 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.667029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.667639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.667918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.668404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.668830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:58.669131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.670543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.680811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.689673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 521/1033] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:58.690953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.699578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485479f-e535-492d-82e4-985480fa8cdb None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.735891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.737288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.737849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.738326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.739282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:58.740640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.742625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.748825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 513/1034] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:58.754346 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.797026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441b36-11f1-4c98-a890-e0e6bd4a923e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.833070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.833776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.834038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.834309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.841501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.842944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:19:58.855052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:58.856045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:19:58.857027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 522/1035] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:19:58.859813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.861804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.868341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0ba77-11c9-40aa-ad5f-22906a82e7a1 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:58.908806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:58.909717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:58.909997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:58.910267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:58.911143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] RBAC: Adding query filter params (type=image) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:58.911447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:58.914110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:58.919370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 514/1036] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:58.925413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:58.925962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:58.971528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f8e72-0fa6-4d2a-a4fa-a5b3429dcd98 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:59.009043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:59.009807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:59.010088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:59.010355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:59.011084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:59.011383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:59.012900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:59.030606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 523/1037] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:59.036112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:59.038424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9e3cc-ae75-4fa5-87c2-15ba3e336b09 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:59.073760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:59.074375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:59.074658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:59.074934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:59.075637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] RBAC: Adding query filter params (interface=public, service_id=c8b2d221e9ec44e1957629bd74ac9617, region_id=RegionOne) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:59.075948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:59.077475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:59.082717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 515/1038] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:19:58 2022] GET /identity/v3/endpoints?service_id=c8b2d221e9ec44e1957629bd74ac9617&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:19:59.088195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:59.114201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:59.148756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f7144-af13-4d08-bdc8-c44e6f1af4ab None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:59.167897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:59.187089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:59.187813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:59.188094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:59.188373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:59.197241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3087243-f0bd-4be4-b70d-a1e353e5c239 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:59.232603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:59.233557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:59.234035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:59.234501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:59.235297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:59.236906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:59.245641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 516/1039] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:19:59 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:19:59.261027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:59.262782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:59.377669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 524/1040] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:19:59 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:19:59.381195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:59.474696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:59.491772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae88303-5a61-4c63-9869-59919e7fc77e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:59.526991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:59.527617 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:59.527896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:59.528166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:59.538902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:59.540340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:19:59.554282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:59.555272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:19:59.556221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:19:59.556221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:19:59.556221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:59.556221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 517/1041] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:59.561078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:59.585872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:59.672706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1cd9b-9dd1-4d4a-9775-6581d6a3f24c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:59.707047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:59.707654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:59.707932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:59.708200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:59.709448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:19:59.709751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:59.711109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:59.723813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 525/1042] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:19:59.730006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:59.830059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806b2a5-8603-4fa4-af6d-f0be3b29221c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:19:59.863552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:19:59.864264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:19:59.864457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:19:59.865244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:19:59.871962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:19:59.873433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:19:59.875159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:19:59.886991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:19:59.887911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:19:59.888844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 518/1043] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:19:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:19:59.895251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:19:59.975096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:19:59.999045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b32c3-6b1e-4192-882f-0541d684d816 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:00.032799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3440e-f21a-44ff-a06f-e8ebae577b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:00.033432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3440e-f21a-44ff-a06f-e8ebae577b57 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:00.033712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3440e-f21a-44ff-a06f-e8ebae577b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:00.033983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3440e-f21a-44ff-a06f-e8ebae577b57 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:00.034673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3440e-f21a-44ff-a06f-e8ebae577b57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:00.034973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3440e-f21a-44ff-a06f-e8ebae577b57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:00.036401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3440e-f21a-44ff-a06f-e8ebae577b57 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:00.048387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 526/1044] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:19:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:00.054706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:00.154608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd56a3-7192-4e93-acc0-e21a5d341ef0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:00.187587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa13616-9c68-4951-bdd0-0da8e5826a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:00.188191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa13616-9c68-4951-bdd0-0da8e5826a4e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:00.188470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa13616-9c68-4951-bdd0-0da8e5826a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:00.188744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa13616-9c68-4951-bdd0-0da8e5826a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:00.189490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa13616-9c68-4951-bdd0-0da8e5826a4e None admin] RBAC: Adding query filter params (role.id=aff6df9c4ba4488ea98690de7d647806, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=536ca83c26354a77b1c5d263c57f4f7d) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:00.189799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa13616-9c68-4951-bdd0-0da8e5826a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:00.191499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa13616-9c68-4951-bdd0-0da8e5826a4e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:00.205301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 519/1045] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:00 2022] GET /identity/v3/role_assignments?user.id=536ca83c26354a77b1c5d263c57f4f7d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=aff6df9c4ba4488ea98690de7d647806 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:00.370377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b516f-413c-413e-8947-f113e704fa4d None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:00.370722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b516f-413c-413e-8947-f113e704fa4d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:00.370901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b516f-413c-413e-8947-f113e704fa4d None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:00.372331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 527/1046] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:00.377423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a649a10-ea39-4fe4-8ce5-aa44aa3a06b0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:00.377707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a649a10-ea39-4fe4-8ce5-aa44aa3a06b0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:00.377976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a649a10-ea39-4fe4-8ce5-aa44aa3a06b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:00.395856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9a649a10-ea39-4fe4-8ce5-aa44aa3a06b0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:00.397058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9a649a10-ea39-4fe4-8ce5-aa44aa3a06b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:00.454401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:00.465941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 520/1047] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:00.489202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b516f-413c-413e-8947-f113e704fa4d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:00.595149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b516f-413c-413e-8947-f113e704fa4d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:00.632647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:00.633228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:00.633540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:00.633813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:00.641220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:00.645166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:00.650615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:00.651723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:00.652744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:00.659795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 528/1048] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:00.661386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a649a10-ea39-4fe4-8ce5-aa44aa3a06b0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:00.761967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a649a10-ea39-4fe4-8ce5-aa44aa3a06b0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:00.796179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:00.796789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:00.797059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:00.797348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:00.798005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:00.798303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:00.799647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:00.812599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 521/1049] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:00.813955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:00.917944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58177d68-489c-4ec1-9d83-863ea81fb3b6 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:00.951500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c367ed3-de56-497a-beda-3ec19bb0833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:00.952131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c367ed3-de56-497a-beda-3ec19bb0833f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:00.952412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c367ed3-de56-497a-beda-3ec19bb0833f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:00.952687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c367ed3-de56-497a-beda-3ec19bb0833f None admin] PATH_INFO: `/v3/users/dd481a445bd74aada724eea90dd7264c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:00.966553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c367ed3-de56-497a-beda-3ec19bb0833f None admin] RBAC: Authorizing `identity:get_user(user_id=dd481a445bd74aada724eea90dd7264c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:00.968066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c367ed3-de56-497a-beda-3ec19bb0833f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:00.974744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 529/1050] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:00 2022] GET /identity/v3/users/dd481a445bd74aada724eea90dd7264c => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:00.976602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:01.033588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:01.078136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab3cdf-c2a7-4e8d-b0af-63de2fce797d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:01.111117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:01.111721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.111995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.112260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.178710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:01.180064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:01.233228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:01.263426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa32c6-78b9-4fce-bffb-9844734fe330 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.263713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa32c6-78b9-4fce-bffb-9844734fe330 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.263990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa32c6-78b9-4fce-bffb-9844734fe330 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.265508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 530/1051] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:01.270335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17aa0e-562d-41a9-aabd-f138da5a7560 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.270644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17aa0e-562d-41a9-aabd-f138da5a7560 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.270960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17aa0e-562d-41a9-aabd-f138da5a7560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.284284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 522/1052] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:01.290241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:01.290880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-1f17aa0e-562d-41a9-aabd-f138da5a7560 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:01.292171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1f17aa0e-562d-41a9-aabd-f138da5a7560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:01.344410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:01.363914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 531/1053] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:20:01.386851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17aa0e-562d-41a9-aabd-f138da5a7560 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:01.390961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4efd5-19a3-4a97-9569-d10de4cd1318 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:01.426536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:01.427335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.427599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.427870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.429170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] RBAC: Authorizing `identity:get_project(project_id=d867e0e841474b2a8e9a5668a6f12b08)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:01.430693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:01.432493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 523/1054] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:01 2022] GET /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:01.438328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:01.462503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:01.492801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17aa0e-562d-41a9-aabd-f138da5a7560 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:01.528386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:01.529127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.529444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.529736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] PATH_INFO: `/v3/services/image` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.530306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:01.531896 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:01.538145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535345f2-f53d-486d-9e7d-6a3c3ea4a5bf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:01.541771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:01.542967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 12:20:01.544002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:01.547899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 532/1055] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 10 12:20:01 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:01.548540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:01.578050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:01.578659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.578940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.579215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.579980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=d867e0e841474b2a8e9a5668a6f12b08, user.id=dd481a445bd74aada724eea90dd7264c) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:01.580292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:01.581767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:01.597302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 524/1056] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:01 2022] GET /identity/v3/role_assignments?user.id=dd481a445bd74aada724eea90dd7264c&scope.project.id=d867e0e841474b2a8e9a5668a6f12b08&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:01.608711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:01.666334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb301f05-bcfd-4512-8c2b-a7719dacfc92 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:01.701773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:01.702399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.702746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.703083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.703795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] RBAC: Adding query filter params (name=image) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:01.704164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:01.705652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:01.716618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 533/1057] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:01.718025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:01.806587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:01.822252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af530383-974f-4c52-9f4a-cef4cc4b3fe3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:01.856099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7cb00-bcb8-4a8b-a58b-8764ab3ea84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:01.856701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cb00-bcb8-4a8b-a58b-8764ab3ea84a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:01.856974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cb00-bcb8-4a8b-a58b-8764ab3ea84a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:01.857242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cb00-bcb8-4a8b-a58b-8764ab3ea84a None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:01.857889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7cb00-bcb8-4a8b-a58b-8764ab3ea84a None admin] RBAC: Adding query filter params (type=image) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:01.858187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7cb00-bcb8-4a8b-a58b-8764ab3ea84a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:01.859541 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7cb00-bcb8-4a8b-a58b-8764ab3ea84a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:01.868998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 525/1058] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:01.870124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:01.971400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3d57b-cd61-42aa-abc5-26796793d93b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:01.985153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:02.006733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cf45e-6677-4bc3-b4d5-e81ca53fb491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:02.007553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf45e-6677-4bc3-b4d5-e81ca53fb491 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:02.008007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf45e-6677-4bc3-b4d5-e81ca53fb491 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:02.008427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf45e-6677-4bc3-b4d5-e81ca53fb491 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:02.009233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cf45e-6677-4bc3-b4d5-e81ca53fb491 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:02.010777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cf45e-6677-4bc3-b4d5-e81ca53fb491 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:02.037116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 534/1059] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:20:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:20:02.191543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb211c-6edc-48df-adf6-9078eeab3c1e None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:02.192053 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb211c-6edc-48df-adf6-9078eeab3c1e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:02.192160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb211c-6edc-48df-adf6-9078eeab3c1e None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:02.194101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 526/1060] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:02.199407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b4ab-51f9-4927-8e5b-70a70ab02376 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:02.199698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b4ab-51f9-4927-8e5b-70a70ab02376 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:02.199973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b4ab-51f9-4927-8e5b-70a70ab02376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:02.219106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8342b4ab-51f9-4927-8e5b-70a70ab02376 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:02.220346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8342b4ab-51f9-4927-8e5b-70a70ab02376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:02.297390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 535/1061] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:02.319387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb211c-6edc-48df-adf6-9078eeab3c1e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:02.422773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb211c-6edc-48df-adf6-9078eeab3c1e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:02.457994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:02.458639 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:02.458912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:02.459177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:02.469096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:02.473743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:02.474706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:02.475553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 527/1062] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 10 12:20:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:02.482951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b4ab-51f9-4927-8e5b-70a70ab02376 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:02.583461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:02.590441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b4ab-51f9-4927-8e5b-70a70ab02376 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:02.630163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:02.630800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:02.631080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:02.631436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:02.632108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:02.632413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:02.633818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:02.646653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 536/1063] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 10 12:20:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:02.648161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:02.662030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:02.756104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f247339-de58-4add-a213-c2e488d9d122 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:02.790001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:02.790625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:02.790898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:02.791164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:02.801357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:02.803410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:20:02.817638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:02.818615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:20:02.819553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:20:02.819553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:02.819553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:02.821597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 528/1064] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:02.824524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:02.926165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed265b-869f-4851-9e2a-64be6e44cbea None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:02.958157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:02.959286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:02.959563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:02.959831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:02.960468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:02.960767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:02.962158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:02.978557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 537/1065] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:02.980634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.044284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:03.085147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc195e-f981-4637-9c2c-9560665bb6e5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:03.118989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:03.119613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.119917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.120192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.127854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:03.129311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:20:03.141025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:03.142156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:20:03.143151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:03.146915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 529/1066] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:20:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:03.148295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.167801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:03.251762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc9b16-f248-492d-b422-d2f1decef5bb None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:03.285879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:03.286989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.287269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.287532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.288252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:03.288555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:03.289995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:03.312363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 538/1067] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:20:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:03.313809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.413179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576d655-aab2-4403-ad2d-375c67387d79 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:03.435382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:03.448047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd6eb3-2932-41a0-a2dc-3e41e83dbd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:03.448667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd6eb3-2932-41a0-a2dc-3e41e83dbd48 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.448940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd6eb3-2932-41a0-a2dc-3e41e83dbd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.449207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd6eb3-2932-41a0-a2dc-3e41e83dbd48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.459649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd6eb3-2932-41a0-a2dc-3e41e83dbd48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:03.461087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd6eb3-2932-41a0-a2dc-3e41e83dbd48 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-80bd6eb3-2932-41a0-a2dc-3e41e83dbd48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:20:03.475345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:03.476308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:20:03.482545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:20:03.482545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:03.482545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:03.482545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 530/1068] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:03.482774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.559922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:03.585353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64c48f-f57b-4dda-81b0-0898ddc2f6e4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:03.611351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff612f-28c2-4b32-bd0d-312c6e0d7125 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.611640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff612f-28c2-4b32-bd0d-312c6e0d7125 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.611917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff612f-28c2-4b32-bd0d-312c6e0d7125 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.613294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 531/1069] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:20:03.618717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf84a7-6da0-48b4-92cb-5af4f4bc8146 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.619079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf84a7-6da0-48b4-92cb-5af4f4bc8146 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.619427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf84a7-6da0-48b4-92cb-5af4f4bc8146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.628341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:03.628955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.629232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.629538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.630209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:03.630510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:03.632006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:03.638444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-fdaf84a7-6da0-48b4-92cb-5af4f4bc8146 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:03.639713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-fdaf84a7-6da0-48b4-92cb-5af4f4bc8146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:03.644165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 539/1070] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:03.650057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.713112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 532/1071] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:20:03.735658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf84a7-6da0-48b4-92cb-5af4f4bc8146 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.758906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae77d9a-c76a-4329-8f1e-1e2522d8150f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:03.795810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:03.796442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.796746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.796993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.799706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:03.805401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:03.807017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:20:03.818302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:03.819387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:03.820363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 540/1072] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:03.825763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.837688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf84a7-6da0-48b4-92cb-5af4f4bc8146 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:03.840491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:03.877499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:03.878188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.878474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.878749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.889154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:03.893831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:03.894768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:03.895625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:03.900497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 533/1073] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:03.901184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:03.933357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da155c20-09c0-48e2-805f-a12f1b7fc8e1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:03.969044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:03.969701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:03.969987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:03.970264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:03.970954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:03.971261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:03.972694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:03.986909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 541/1074] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:03.992524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:04.008244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04812-5cbd-4230-ba30-dd7845798fb4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.042757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.043360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.043651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.043924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.044574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:04.044883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.046280 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.051232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:04.055416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 534/1075] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:04.061067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:04.089178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:04.100556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16917c9-b4f2-45db-af35-12933f8910d1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.134574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd404b5-4c25-4c5d-b906-0d8f62a6200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.135230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd404b5-4c25-4c5d-b906-0d8f62a6200f None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.135504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd404b5-4c25-4c5d-b906-0d8f62a6200f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.135772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd404b5-4c25-4c5d-b906-0d8f62a6200f None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/536ca83c26354a77b1c5d263c57f4f7d/roles/aff6df9c4ba4488ea98690de7d647806` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.137903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd404b5-4c25-4c5d-b906-0d8f62a6200f None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=536ca83c26354a77b1c5d263c57f4f7d, role_id=aff6df9c4ba4488ea98690de7d647806)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.139331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd404b5-4c25-4c5d-b906-0d8f62a6200f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.152351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 542/1076] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:20:03 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/536ca83c26354a77b1c5d263c57f4f7d/roles/aff6df9c4ba4488ea98690de7d647806 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:20:04.153549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448aad7-799b-462d-bfdd-851f18629d0b None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.153834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448aad7-799b-462d-bfdd-851f18629d0b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.154107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448aad7-799b-462d-bfdd-851f18629d0b None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.155533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 543/1077] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:04.161193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894841-273d-40df-912a-2570fd4bf597 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.161663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894841-273d-40df-912a-2570fd4bf597 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.161945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894841-273d-40df-912a-2570fd4bf597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.165623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6607d-9a31-4867-aee8-c9942f496741 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.180681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-54894841-273d-40df-912a-2570fd4bf597 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:04.181964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-54894841-273d-40df-912a-2570fd4bf597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:04.204275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.204938 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.205221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.205534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] PATH_INFO: `/v3/users/dd481a445bd74aada724eea90dd7264c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.206785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] RBAC: Authorizing `identity:get_user(user_id=dd481a445bd74aada724eea90dd7264c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.208327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.210651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 535/1078] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:04 2022] GET /identity/v3/users/dd481a445bd74aada724eea90dd7264c => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:04.216756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:04.273931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 544/1079] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:20:04.296761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894841-273d-40df-912a-2570fd4bf597 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:04.332381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:04.340492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c988-e4e7-4aef-89be-cf61d71353b3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.351994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:04.379281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.379281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.379594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.379822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.381495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] RBAC: Authorizing `identity:get_project(project_id=d867e0e841474b2a8e9a5668a6f12b08)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.382934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.384799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 536/1080] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:04 2022] GET /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:04.390503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:04.412315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894841-273d-40df-912a-2570fd4bf597 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.447416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.448177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.448540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.448882 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.495576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332df46-1b02-4672-bbc2-ebbee47e74fb None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.522581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.524038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.533421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.534074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.534350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.534610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08/users/dd481a445bd74aada724eea90dd7264c/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.537136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] RBAC: Authorizing `identity:create_grant(project_id=d867e0e841474b2a8e9a5668a6f12b08, user_id=dd481a445bd74aada724eea90dd7264c, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.538580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.552577 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 537/1081] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:20:04 2022] PUT /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08/users/dd481a445bd74aada724eea90dd7264c/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:20:04.575616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:04.650110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 545/1082] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:20:04.651382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:04.678915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:04.763753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba82c50-8548-41c4-84c5-bc630adc3baf None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.797643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa6043-1cff-4404-89ed-d2b25eab2e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.798434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6043-1cff-4404-89ed-d2b25eab2e81 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.798818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6043-1cff-4404-89ed-d2b25eab2e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.799186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6043-1cff-4404-89ed-d2b25eab2e81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.809981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6043-1cff-4404-89ed-d2b25eab2e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.811529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6043-1cff-4404-89ed-d2b25eab2e81 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-6cfa6043-1cff-4404-89ed-d2b25eab2e81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:20:04.826003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:04.827152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:20:04.828274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:20:04.828274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:04.828274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:04.829556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 538/1083] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:04.835895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:04.936064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565195-c579-40a1-a141-1a2d8bbf8f5b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:04.969947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480166f-62e2-44a9-9028-d28c771ed80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:04.970684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480166f-62e2-44a9-9028-d28c771ed80a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:04.971082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480166f-62e2-44a9-9028-d28c771ed80a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:04.971464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480166f-62e2-44a9-9028-d28c771ed80a None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:04.972187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480166f-62e2-44a9-9028-d28c771ed80a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:04.972585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480166f-62e2-44a9-9028-d28c771ed80a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:04.974091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480166f-62e2-44a9-9028-d28c771ed80a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:04.993311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 546/1084] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:06.157060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dfb2-34b1-4add-86c4-779a7f271e34 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.157608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dfb2-34b1-4add-86c4-779a7f271e34 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.157724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dfb2-34b1-4add-86c4-779a7f271e34 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.159005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 539/1085] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:06.164126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21610296-a437-4a28-ba61-ac18349684b2 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.164548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21610296-a437-4a28-ba61-ac18349684b2 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.164664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21610296-a437-4a28-ba61-ac18349684b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.178580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:06.183901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-21610296-a437-4a28-ba61-ac18349684b2 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:06.185060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-21610296-a437-4a28-ba61-ac18349684b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:06.249243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 547/1086] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:06.272734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dfb2-34b1-4add-86c4-779a7f271e34 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:06.347409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:06.374381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dfb2-34b1-4add-86c4-779a7f271e34 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:06.406111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:06.406728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.407030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.407360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.417502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:06.422093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:06.423141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 12:20:06.424180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:06.425425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 540/1087] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 10 12:20:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:06.429864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21610296-a437-4a28-ba61-ac18349684b2 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:06.524699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21610296-a437-4a28-ba61-ac18349684b2 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:06.558006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536ad91-1071-4389-b8f8-8c3eadbb413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:06.558611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ad91-1071-4389-b8f8-8c3eadbb413b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.558887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ad91-1071-4389-b8f8-8c3eadbb413b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.559156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ad91-1071-4389-b8f8-8c3eadbb413b None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.559814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536ad91-1071-4389-b8f8-8c3eadbb413b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:06.560113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536ad91-1071-4389-b8f8-8c3eadbb413b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:06.561516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536ad91-1071-4389-b8f8-8c3eadbb413b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:06.563617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:06.570619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 548/1088] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 10 12:20:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:06.576366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:06.586194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc793449-6965-4b05-b82e-945821747e2c None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.586479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc793449-6965-4b05-b82e-945821747e2c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.586750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc793449-6965-4b05-b82e-945821747e2c None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.588072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 549/1089] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:20:06.592828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeae11-1fe4-446f-8684-6f7e936c7e8d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.593171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeae11-1fe4-446f-8684-6f7e936c7e8d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.593563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeae11-1fe4-446f-8684-6f7e936c7e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.611600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c3eeae11-1fe4-446f-8684-6f7e936c7e8d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:06.612770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c3eeae11-1fe4-446f-8684-6f7e936c7e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:06.677453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 550/1090] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:20:06.678277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ab7fe-301d-461d-86f0-ceb201cf919f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:06.699730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeae11-1fe4-446f-8684-6f7e936c7e8d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:06.711726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:06.712350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.712628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.712895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.714785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:06.723563 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:06.725615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:20:06.740720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:06.741828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:20:06.742731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:20:06.742731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:06.742731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:06.742731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 541/1091] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:06.747526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:06.798359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeae11-1fe4-446f-8684-6f7e936c7e8d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:06.823714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:06.832075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:06.832704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.832975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.833783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.843497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:06.845203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525c1c4-8189-4d9c-b6d5-dca65572f83e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:06.848126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:06.849914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:06.850928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 551/1092] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:06.855852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:06.884850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:06.885600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.885970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.886317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.887021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:06.887393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:06.888837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:06.900741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 542/1093] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:06.906293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:06.951681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:06.956518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30fb48-2a46-4a0a-ab32-05507903102e None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:06.990092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:06.990681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:06.990957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:06.991229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:06.991881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:06.992179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:06.993573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.001543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 552/1094] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:07.003897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512013c-24c1-40c4-9287-b425feec1801 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.007777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.038189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.038844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.039128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.039446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.046390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.047819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.056439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:20:07.059793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:07.060933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:07.062119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 543/1095] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 10 12:20:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:07.066478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.108139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb34c31-a284-4d2f-bdce-94bdff44f1f1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.142102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.142729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.143010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.143287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] PATH_INFO: `/v3/users/dd481a445bd74aada724eea90dd7264c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.144423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] RBAC: Authorizing `identity:get_user(user_id=dd481a445bd74aada724eea90dd7264c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.145923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.147500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 553/1096] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/users/dd481a445bd74aada724eea90dd7264c => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:07.153092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.163887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3df6f5-107e-499f-890c-90951f0257f4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.171478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.199193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.199793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.200123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.200346 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.201079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:07.201425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.202903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.220225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 544/1097] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:07.225895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.252444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f06f81-4c0c-401f-b905-064f3c1125af None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.269242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.286594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.287195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.287467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.287731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.324384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd8fb4-225f-47d0-9f3e-58213132d3b3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.353127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.354576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.359667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.360292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.360571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.360847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.392123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.428622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.430118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.450602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 554/1098] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:20:07.456503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.474166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.526214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 545/1099] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:20:07.531785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.554341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b75cb-a37b-4f24-9a52-645c888d52ef None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.589304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.589959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.590238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.590537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.594010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.594136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] RBAC: Authorizing `identity:get_project(project_id=d867e0e841474b2a8e9a5668a6f12b08)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.594136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.594921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 555/1100] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:07.600662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.632280 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e9172-9f54-49db-a6ca-5f9510fa24d5 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.665633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.666252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.666525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.666795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.676560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.678567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.684082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:20:07.693950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:07.695098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:20:07.696197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:20:07.696197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:20:07.696197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:07.696197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 546/1101] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:07.701032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.702382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b798c-6454-432b-8e96-482fdabe2e41 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.736371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.736908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.737188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.737488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.738210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=d867e0e841474b2a8e9a5668a6f12b08, user.id=dd481a445bd74aada724eea90dd7264c) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:07.738518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.740471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.746334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 556/1102] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/role_assignments?user.id=dd481a445bd74aada724eea90dd7264c&scope.project.id=d867e0e841474b2a8e9a5668a6f12b08&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:07.803339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0b91c-859a-45f0-b37d-d8e7d0740d1b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:07.820744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:07.839787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:07.840407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:07.840686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:07.840963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:07.841653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:07.841962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:07.843374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:07.857012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 547/1103] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:07.862884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:07.970654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfccc-9e56-4586-a16d-ef9231789707 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:08.004518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:08.005081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:08.005409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:08.005681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:08.012177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:08.013648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:20:08.025277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:08.026294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:08.027399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 557/1104] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:08.032204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:08.130264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5b1d2-0836-44c0-8ab6-4f30058b0b1f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:08.163300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7599e-fd61-4da1-9e61-547262d59b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:08.163889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7599e-fd61-4da1-9e61-547262d59b92 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:08.164162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7599e-fd61-4da1-9e61-547262d59b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:08.164682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7599e-fd61-4da1-9e61-547262d59b92 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:08.165093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7599e-fd61-4da1-9e61-547262d59b92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:08.165427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7599e-fd61-4da1-9e61-547262d59b92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:08.166875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7599e-fd61-4da1-9e61-547262d59b92 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:08.178062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 548/1105] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:08.184436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:08.197106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:08.282861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc08f2f-2c37-47f2-a548-0f5398a6df97 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:08.316206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56e415-faaa-4fd3-adbc-7d1bb032bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:08.316809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56e415-faaa-4fd3-adbc-7d1bb032bf14 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:08.317086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56e415-faaa-4fd3-adbc-7d1bb032bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:08.317392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56e415-faaa-4fd3-adbc-7d1bb032bf14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:08.318179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56e415-faaa-4fd3-adbc-7d1bb032bf14 None admin] RBAC: Adding query filter params (role.id=aff6df9c4ba4488ea98690de7d647806, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=536ca83c26354a77b1c5d263c57f4f7d) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:08.318490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56e415-faaa-4fd3-adbc-7d1bb032bf14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:08.319925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56e415-faaa-4fd3-adbc-7d1bb032bf14 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:08.325628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 558/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:08 2022] GET /identity/v3/role_assignments?user.id=536ca83c26354a77b1c5d263c57f4f7d&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=aff6df9c4ba4488ea98690de7d647806 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:09.779511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bfad9-10d0-49a6-b829-e06b5ffb2e1f None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:09.780021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bfad9-10d0-49a6-b829-e06b5ffb2e1f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:09.780129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bfad9-10d0-49a6-b829-e06b5ffb2e1f None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:09.781464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 549/1107] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:09.787071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f38d8-25f8-4374-90b5-f9bcb223725c None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:09.787474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f38d8-25f8-4374-90b5-f9bcb223725c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:09.787612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f38d8-25f8-4374-90b5-f9bcb223725c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:09.805148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7c7f38d8-25f8-4374-90b5-f9bcb223725c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:09.806338 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7c7f38d8-25f8-4374-90b5-f9bcb223725c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:09.869546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 559/1108] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:09.892880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bfad9-10d0-49a6-b829-e06b5ffb2e1f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:09.929490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:09.997022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bfad9-10d0-49a6-b829-e06b5ffb2e1f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:10.029999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c4323-6fab-48c3-b817-16ba112944d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:10.030613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c4323-6fab-48c3-b817-16ba112944d7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:10.030890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c4323-6fab-48c3-b817-16ba112944d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:10.031164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c4323-6fab-48c3-b817-16ba112944d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:10.032260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c4323-6fab-48c3-b817-16ba112944d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:10.033739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c4323-6fab-48c3-b817-16ba112944d7 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:10.035285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 550/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:10.040930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f38d8-25f8-4374-90b5-f9bcb223725c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:10.043973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:10.135524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f38d8-25f8-4374-90b5-f9bcb223725c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:10.167611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2db31e-58ff-431e-b216-a3dbff62d49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:10.168195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2db31e-58ff-431e-b216-a3dbff62d49d None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:10.168471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2db31e-58ff-431e-b216-a3dbff62d49d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:10.168739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2db31e-58ff-431e-b216-a3dbff62d49d None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:10.169395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2db31e-58ff-431e-b216-a3dbff62d49d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:10.170813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2db31e-58ff-431e-b216-a3dbff62d49d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:10.190420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 560/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:20:10 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:20:12.187512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3c47a-aa35-4635-9059-7fcba26e3c1f None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:12.188034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3c47a-aa35-4635-9059-7fcba26e3c1f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:12.188034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3c47a-aa35-4635-9059-7fcba26e3c1f None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:12.189445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 551/1111] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:12.195026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cbb65-1f0f-4783-b739-5f4a5fba30f6 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:12.195414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cbb65-1f0f-4783-b739-5f4a5fba30f6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:12.195567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cbb65-1f0f-4783-b739-5f4a5fba30f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:12.212924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-441cbb65-1f0f-4783-b739-5f4a5fba30f6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:12.214126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-441cbb65-1f0f-4783-b739-5f4a5fba30f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:12.267517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:12.279285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 561/1112] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:12.302905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3c47a-aa35-4635-9059-7fcba26e3c1f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:12.401552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3c47a-aa35-4635-9059-7fcba26e3c1f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:12.403815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:12.435291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:12.435903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:12.436181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:12.436451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] PATH_INFO: `/v3/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:12.437899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] RBAC: Authorizing `identity:get_role(role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:12.439201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:12.440516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 552/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:12 2022] GET /identity/v3/roles/807306af822540d7986dfb93d0b3f344 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:12.446407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cbb65-1f0f-4783-b739-5f4a5fba30f6 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:12.543440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cbb65-1f0f-4783-b739-5f4a5fba30f6 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:12.576187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:12.576811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:12.577092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:12.577396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] PATH_INFO: `/v3/users/596aae3a9a1d4f2bb2d216802b8c7039` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:12.590253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] RBAC: Authorizing `identity:get_user(user_id=596aae3a9a1d4f2bb2d216802b8c7039)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:12.591726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:12.593353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 562/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:12 2022] GET /identity/v3/users/596aae3a9a1d4f2bb2d216802b8c7039 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:12.599581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:12.696280 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e685b3e-cad2-4036-a5f4-630af91ac89d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:12.729181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:12.729799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:12.730079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:12.730352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:12.768567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:12.795233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:12.796662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:12.889535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 553/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:12.895302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:12.944154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:12.993440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bef79-df0b-418d-8dba-4bd58377f5b0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:13.026519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de9a13-0119-487d-b4e1-a4313ce9b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:13.027137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de9a13-0119-487d-b4e1-a4313ce9b088 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:13.027415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de9a13-0119-487d-b4e1-a4313ce9b088 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:13.027687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de9a13-0119-487d-b4e1-a4313ce9b088 None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:13.029050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de9a13-0119-487d-b4e1-a4313ce9b088 None admin] RBAC: Authorizing `identity:get_project(project_id=d867e0e841474b2a8e9a5668a6f12b08)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:13.030514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de9a13-0119-487d-b4e1-a4313ce9b088 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:13.032120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 563/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:12 2022] GET /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:13.038059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:13.104141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:13.132742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d715a5-c167-491b-9e66-31d592cfe973 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:13.165300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92b96c-feb8-475f-8985-0ae428c7bd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:13.165893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92b96c-feb8-475f-8985-0ae428c7bd40 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:13.166177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92b96c-feb8-475f-8985-0ae428c7bd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:13.166448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92b96c-feb8-475f-8985-0ae428c7bd40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:13.167159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92b96c-feb8-475f-8985-0ae428c7bd40 None admin] RBAC: Adding query filter params (role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=d867e0e841474b2a8e9a5668a6f12b08, user.id=596aae3a9a1d4f2bb2d216802b8c7039) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:13.167464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92b96c-feb8-475f-8985-0ae428c7bd40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:13.168871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92b96c-feb8-475f-8985-0ae428c7bd40 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:13.174065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 554/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:13 2022] GET /identity/v3/role_assignments?user.id=596aae3a9a1d4f2bb2d216802b8c7039&scope.project.id=d867e0e841474b2a8e9a5668a6f12b08&role.id=807306af822540d7986dfb93d0b3f344 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:15.116523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342293d2-4341-48e4-93cc-c4e9602bb8c8 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:15.117040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342293d2-4341-48e4-93cc-c4e9602bb8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:15.117040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342293d2-4341-48e4-93cc-c4e9602bb8c8 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:15.118437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 564/1118] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:15.123546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e505-6a95-4058-ae07-302797f9823d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:15.123916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e505-6a95-4058-ae07-302797f9823d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:15.124087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e505-6a95-4058-ae07-302797f9823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:15.142711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f997e505-6a95-4058-ae07-302797f9823d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:15.143892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f997e505-6a95-4058-ae07-302797f9823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:15.206811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 555/1119] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:15.228187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342293d2-4341-48e4-93cc-c4e9602bb8c8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:15.324890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342293d2-4341-48e4-93cc-c4e9602bb8c8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:15.340310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:15.358097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:15.358701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:15.358975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:15.359265 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] PATH_INFO: `/v3/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:15.360590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] RBAC: Authorizing `identity:get_role(role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:15.361968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:15.363250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 565/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:15 2022] GET /identity/v3/roles/807306af822540d7986dfb93d0b3f344 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:15.369503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997e505-6a95-4058-ae07-302797f9823d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:15.431074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:15.463436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997e505-6a95-4058-ae07-302797f9823d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:15.496467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:15.497071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:15.497372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:15.497642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] PATH_INFO: `/v3/users/596aae3a9a1d4f2bb2d216802b8c7039` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:15.498723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] RBAC: Authorizing `identity:get_user(user_id=596aae3a9a1d4f2bb2d216802b8c7039)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:15.500132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:15.501700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 556/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:15 2022] GET /identity/v3/users/596aae3a9a1d4f2bb2d216802b8c7039 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:15.507442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:15.605082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ba04-6f77-4f84-87da-4d4b27ddcb33 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:15.637551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfaf763-7319-4ae3-a9e5-36e04bd0cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:15.638104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaf763-7319-4ae3-a9e5-36e04bd0cd8b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:15.638378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaf763-7319-4ae3-a9e5-36e04bd0cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:15.638643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaf763-7319-4ae3-a9e5-36e04bd0cd8b None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:15.639724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfaf763-7319-4ae3-a9e5-36e04bd0cd8b None admin] RBAC: Authorizing `identity:get_project(project_id=d867e0e841474b2a8e9a5668a6f12b08)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:15.641149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfaf763-7319-4ae3-a9e5-36e04bd0cd8b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:15.642782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 566/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:15 2022] GET /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:15.649202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:15.749372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f9a61-81b4-47d4-92c6-e9efbacffc77 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:15.781008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471de0c4-ba2a-4c1b-a0f6-bc677f0f1d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:15.781667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de0c4-ba2a-4c1b-a0f6-bc677f0f1d7b None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:15.781943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de0c4-ba2a-4c1b-a0f6-bc677f0f1d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:15.782211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de0c4-ba2a-4c1b-a0f6-bc677f0f1d7b None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08/users/596aae3a9a1d4f2bb2d216802b8c7039/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:15.784279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471de0c4-ba2a-4c1b-a0f6-bc677f0f1d7b None admin] RBAC: Authorizing `identity:create_grant(project_id=d867e0e841474b2a8e9a5668a6f12b08, user_id=596aae3a9a1d4f2bb2d216802b8c7039, role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:15.785746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471de0c4-ba2a-4c1b-a0f6-bc677f0f1d7b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:15.788473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:15.798753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 557/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:20:15 2022] PUT /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08/users/596aae3a9a1d4f2bb2d216802b8c7039/roles/807306af822540d7986dfb93d0b3f344 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:20:17.730970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61c850-15b7-4100-980d-2787fefe1876 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:17.731477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61c850-15b7-4100-980d-2787fefe1876 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:17.731566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61c850-15b7-4100-980d-2787fefe1876 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:17.732878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 567/1124] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:17.738022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b246-c071-4cef-9a87-9711c9622f74 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:17.738419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b246-c071-4cef-9a87-9711c9622f74 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:17.738576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b246-c071-4cef-9a87-9711c9622f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:17.756691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9858b246-c071-4cef-9a87-9711c9622f74 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:17.757897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9858b246-c071-4cef-9a87-9711c9622f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:17.821800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 558/1125] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:17.844243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61c850-15b7-4100-980d-2787fefe1876 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:17.897348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:17.943230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61c850-15b7-4100-980d-2787fefe1876 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:17.975928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:17.976545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:17.976823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:17.977365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] PATH_INFO: `/v3/roles/807306af822540d7986dfb93d0b3f344` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:17.978755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] RBAC: Authorizing `identity:get_role(role_id=807306af822540d7986dfb93d0b3f344)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:17.980116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:17.981437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 568/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:17 2022] GET /identity/v3/roles/807306af822540d7986dfb93d0b3f344 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:17.987104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b246-c071-4cef-9a87-9711c9622f74 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:18.082190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b246-c071-4cef-9a87-9711c9622f74 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:18.107071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:18.118271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:18.118895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:18.119172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:18.119442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] PATH_INFO: `/v3/users/596aae3a9a1d4f2bb2d216802b8c7039` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:18.120546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] RBAC: Authorizing `identity:get_user(user_id=596aae3a9a1d4f2bb2d216802b8c7039)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:18.122025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:18.123610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 559/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:17 2022] GET /identity/v3/users/596aae3a9a1d4f2bb2d216802b8c7039 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:18.129502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:18.223409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad942148-515e-4870-94ef-4068fa53263b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:18.243491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:18.257630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:18.258235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:18.258510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:18.258779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:18.322779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:18.324198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:18.418147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 569/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:18.424122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:18.521089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c01c6-73c1-4eb8-aadb-cd03369f8638 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:18.552552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73ed78-da8e-4fe3-ab18-665e7a5ebadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:18.553159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73ed78-da8e-4fe3-ab18-665e7a5ebadf None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:18.553464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73ed78-da8e-4fe3-ab18-665e7a5ebadf None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:18.553740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73ed78-da8e-4fe3-ab18-665e7a5ebadf None admin] PATH_INFO: `/v3/projects/d867e0e841474b2a8e9a5668a6f12b08` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:18.554836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73ed78-da8e-4fe3-ab18-665e7a5ebadf None admin] RBAC: Authorizing `identity:get_project(project_id=d867e0e841474b2a8e9a5668a6f12b08)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:18.556277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73ed78-da8e-4fe3-ab18-665e7a5ebadf None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:18.557899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 560/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:18 2022] GET /identity/v3/projects/d867e0e841474b2a8e9a5668a6f12b08 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:18.563801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:18.586019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:18.662464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf40c0-651d-41f9-aff7-4e5ccb9e3791 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:18.695641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300aadea-3dcd-4f3d-a927-a89957141567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:18.696244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aadea-3dcd-4f3d-a927-a89957141567 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:18.696520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aadea-3dcd-4f3d-a927-a89957141567 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:18.696790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300aadea-3dcd-4f3d-a927-a89957141567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:18.697540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300aadea-3dcd-4f3d-a927-a89957141567 None admin] RBAC: Adding query filter params (role.id=807306af822540d7986dfb93d0b3f344, scope.project.id=d867e0e841474b2a8e9a5668a6f12b08, user.id=596aae3a9a1d4f2bb2d216802b8c7039) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:18.697850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300aadea-3dcd-4f3d-a927-a89957141567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:18.699264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300aadea-3dcd-4f3d-a927-a89957141567 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:18.704837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 570/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:18 2022] GET /identity/v3/role_assignments?user.id=596aae3a9a1d4f2bb2d216802b8c7039&scope.project.id=d867e0e841474b2a8e9a5668a6f12b08&role.id=807306af822540d7986dfb93d0b3f344 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:20.696284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b7fc-08db-41c6-8bdb-a0320e525683 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:20.696835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b7fc-08db-41c6-8bdb-a0320e525683 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:20.696835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b7fc-08db-41c6-8bdb-a0320e525683 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:20.698211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 561/1131] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:20.703237 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452563-48a5-4d5a-9845-db1cffb6eaba None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:20.703677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452563-48a5-4d5a-9845-db1cffb6eaba None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:20.703795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452563-48a5-4d5a-9845-db1cffb6eaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:20.721976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7a452563-48a5-4d5a-9845-db1cffb6eaba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:20.723161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7a452563-48a5-4d5a-9845-db1cffb6eaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:20.787964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 571/1132] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:20.810326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b7fc-08db-41c6-8bdb-a0320e525683 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:20.870211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:20.910560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b7fc-08db-41c6-8bdb-a0320e525683 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:20.943561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d331a6-57ef-4adf-9b08-5df832c84eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:20.944174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d331a6-57ef-4adf-9b08-5df832c84eca None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:20.944449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d331a6-57ef-4adf-9b08-5df832c84eca None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:20.944747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d331a6-57ef-4adf-9b08-5df832c84eca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:20.945867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d331a6-57ef-4adf-9b08-5df832c84eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:20.947285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d331a6-57ef-4adf-9b08-5df832c84eca None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:20.948823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 562/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:20:20.954481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a452563-48a5-4d5a-9845-db1cffb6eaba None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:20.968890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:21.048826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a452563-48a5-4d5a-9845-db1cffb6eaba None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:21.081490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc205a-55f4-46de-bea9-dcad70b47f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:21.082076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc205a-55f4-46de-bea9-dcad70b47f63 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:21.082351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc205a-55f4-46de-bea9-dcad70b47f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:21.082618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc205a-55f4-46de-bea9-dcad70b47f63 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:21.083281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc205a-55f4-46de-bea9-dcad70b47f63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:21.084695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc205a-55f4-46de-bea9-dcad70b47f63 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:21.110097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 572/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:20:20 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:20:23.095659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d731-acb6-45af-9bec-cd502e99cd23 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:23.096224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d731-acb6-45af-9bec-cd502e99cd23 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:23.096224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d731-acb6-45af-9bec-cd502e99cd23 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:23.097592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 563/1135] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:23.103635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def12ca8-a1d1-4411-8a84-4433a5c42fae None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:23.104036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def12ca8-a1d1-4411-8a84-4433a5c42fae None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:23.104187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def12ca8-a1d1-4411-8a84-4433a5c42fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:23.122355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-def12ca8-a1d1-4411-8a84-4433a5c42fae None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:23.123520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-def12ca8-a1d1-4411-8a84-4433a5c42fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:23.180551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:23.188509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 573/1136] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:23.212552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6d731-acb6-45af-9bec-cd502e99cd23 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:23.313040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6d731-acb6-45af-9bec-cd502e99cd23 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:23.338825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:23.346905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f479ef-ef13-4dac-8c03-0cf7be8ab6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:23.347457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f479ef-ef13-4dac-8c03-0cf7be8ab6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:23.347735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f479ef-ef13-4dac-8c03-0cf7be8ab6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:23.348007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f479ef-ef13-4dac-8c03-0cf7be8ab6d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:23.349717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f479ef-ef13-4dac-8c03-0cf7be8ab6d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:23.351153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f479ef-ef13-4dac-8c03-0cf7be8ab6d8 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:23.353011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 564/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:20:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:23.358655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def12ca8-a1d1-4411-8a84-4433a5c42fae None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:23.453505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def12ca8-a1d1-4411-8a84-4433a5c42fae None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:23.486985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72337167-9a3a-4bf2-a51a-c3d7dcc7d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:23.487577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72337167-9a3a-4bf2-a51a-c3d7dcc7d343 None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:23.487856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72337167-9a3a-4bf2-a51a-c3d7dcc7d343 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:23.488125 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72337167-9a3a-4bf2-a51a-c3d7dcc7d343 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:23.488769 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72337167-9a3a-4bf2-a51a-c3d7dcc7d343 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:23.490243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72337167-9a3a-4bf2-a51a-c3d7dcc7d343 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:23.511242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 574/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:20:23 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:20:25.517373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86079-32e9-4b4e-8633-ec101429e45e None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:25.517929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86079-32e9-4b4e-8633-ec101429e45e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:25.517929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86079-32e9-4b4e-8633-ec101429e45e None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:25.519270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 565/1139] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:25.524432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e26e30-df10-4cff-976c-06757171dc71 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:25.524866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e26e30-df10-4cff-976c-06757171dc71 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:25.524975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e26e30-df10-4cff-976c-06757171dc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:25.542384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-50e26e30-df10-4cff-976c-06757171dc71 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:25.543546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-50e26e30-df10-4cff-976c-06757171dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:25.590368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:25.611169 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 575/1140] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:25.634657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86079-32e9-4b4e-8633-ec101429e45e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:25.732471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86079-32e9-4b4e-8633-ec101429e45e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:25.765700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:25.766307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:25.766588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:25.766860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:25.776834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:25.781512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:25.782685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:25.783771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 566/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:25.789157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e26e30-df10-4cff-976c-06757171dc71 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:25.883662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e26e30-df10-4cff-976c-06757171dc71 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:25.915559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:25.916146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:25.916419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:25.916687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:25.917405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:25.917695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:25.919044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:25.927248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 576/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:25.932776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:25.987103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:26.030864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3602f34-72ff-40af-acd6-e72a323acdec None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:26.067171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:26.067791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:26.068069 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:26.068342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] PATH_INFO: `/v3/users/6f4efcc3ffa6454b989815407f430e72` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:26.081166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] RBAC: Authorizing `identity:get_user(user_id=6f4efcc3ffa6454b989815407f430e72)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:26.082631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:26.084231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 567/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:25 2022] GET /identity/v3/users/6f4efcc3ffa6454b989815407f430e72 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:26.090024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:26.135270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:26.187731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368a44-e98f-4f4c-bf89-c1138a142df0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:26.219386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:26.219974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:26.220249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:26.220517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:26.285747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:26.287745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:26.329255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:26.383023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 577/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:26.388883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:26.485636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2b7b5-5cd5-41ae-bba1-3a266f664b50 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:26.499598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:26.519488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb4c33-0a18-492b-812b-72ff6267c1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:26.520089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb4c33-0a18-492b-812b-72ff6267c1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:26.520961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb4c33-0a18-492b-812b-72ff6267c1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:26.521244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb4c33-0a18-492b-812b-72ff6267c1c2 None admin] PATH_INFO: `/v3/projects/0a5c592379884eb492794b3d316145b0` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:26.522378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb4c33-0a18-492b-812b-72ff6267c1c2 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5c592379884eb492794b3d316145b0)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:26.523816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb4c33-0a18-492b-812b-72ff6267c1c2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:26.525458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 568/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:26 2022] GET /identity/v3/projects/0a5c592379884eb492794b3d316145b0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:26.531344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:26.625953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944ee43-836a-425b-98a8-cd32a29e3239 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:26.660153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe65c09d-988c-4e06-b1bd-d0d107df1e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:26.660745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65c09d-988c-4e06-b1bd-d0d107df1e05 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:26.661050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65c09d-988c-4e06-b1bd-d0d107df1e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:26.661348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65c09d-988c-4e06-b1bd-d0d107df1e05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:26.662066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65c09d-988c-4e06-b1bd-d0d107df1e05 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=0a5c592379884eb492794b3d316145b0, user.id=6f4efcc3ffa6454b989815407f430e72) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:26.662377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65c09d-988c-4e06-b1bd-d0d107df1e05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:26.663800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe65c09d-988c-4e06-b1bd-d0d107df1e05 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:26.668939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 578/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:26 2022] GET /identity/v3/role_assignments?user.id=6f4efcc3ffa6454b989815407f430e72&scope.project.id=0a5c592379884eb492794b3d316145b0&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:28.615332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989563cc-d1e3-43f8-875f-f48d2f7486bc None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:28.615806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989563cc-d1e3-43f8-875f-f48d2f7486bc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:28.615922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989563cc-d1e3-43f8-875f-f48d2f7486bc None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:28.617216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 569/1147] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:28.622456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c82f1-14f6-4a98-88c0-a83ab076567a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:28.622870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c82f1-14f6-4a98-88c0-a83ab076567a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:28.622998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c82f1-14f6-4a98-88c0-a83ab076567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:28.641048 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-dc5c82f1-14f6-4a98-88c0-a83ab076567a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:28.642240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-dc5c82f1-14f6-4a98-88c0-a83ab076567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:28.644111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:28.708177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 579/1148] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:28.729782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989563cc-d1e3-43f8-875f-f48d2f7486bc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:28.826467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989563cc-d1e3-43f8-875f-f48d2f7486bc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:28.860949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:28.861525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:28.861798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:28.862062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:28.872518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:28.877158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:28.878130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:28.879098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:28.883359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 570/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:28.884358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c82f1-14f6-4a98-88c0-a83ab076567a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:28.984983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c82f1-14f6-4a98-88c0-a83ab076567a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:29.018488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:29.019088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:29.019363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:29.019628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:29.020306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:29.020603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:29.021994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:29.026938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:29.034652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 580/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:29.037039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:29.080740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:29.136607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6168bd-80ba-4bc9-a8be-c05cf18cbda9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:29.169763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:29.170384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:29.170664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:29.170937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] PATH_INFO: `/v3/users/6f4efcc3ffa6454b989815407f430e72` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:29.172051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=6f4efcc3ffa6454b989815407f430e72)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:29.173511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:29.175086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 571/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:29 2022] GET /identity/v3/users/6f4efcc3ffa6454b989815407f430e72 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:29.180893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:29.276739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0be7f2-7086-443e-ab9c-38e749ffa004 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:29.309222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e445256-d2fc-4338-b32d-2e77f1938fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:29.309862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445256-d2fc-4338-b32d-2e77f1938fcc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:29.310139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445256-d2fc-4338-b32d-2e77f1938fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:29.310407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445256-d2fc-4338-b32d-2e77f1938fcc None admin] PATH_INFO: `/v3/projects/0a5c592379884eb492794b3d316145b0` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:29.311500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e445256-d2fc-4338-b32d-2e77f1938fcc None admin] RBAC: Authorizing `identity:get_project(project_id=0a5c592379884eb492794b3d316145b0)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:29.312906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e445256-d2fc-4338-b32d-2e77f1938fcc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:29.314524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 581/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:29 2022] GET /identity/v3/projects/0a5c592379884eb492794b3d316145b0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:29.321002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:29.417508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3cac2-d442-403d-81a0-25b8f09d0fb0 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:29.427851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:29.451211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334387b-dda8-49d8-b885-3de15097ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:29.451850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334387b-dda8-49d8-b885-3de15097ba54 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:29.452132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334387b-dda8-49d8-b885-3de15097ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:29.452402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334387b-dda8-49d8-b885-3de15097ba54 None admin] PATH_INFO: `/v3/projects/0a5c592379884eb492794b3d316145b0/users/6f4efcc3ffa6454b989815407f430e72/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:29.454304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334387b-dda8-49d8-b885-3de15097ba54 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a5c592379884eb492794b3d316145b0, user_id=6f4efcc3ffa6454b989815407f430e72, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:29.455728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334387b-dda8-49d8-b885-3de15097ba54 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:29.468909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 572/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:20:29 2022] PUT /identity/v3/projects/0a5c592379884eb492794b3d316145b0/users/6f4efcc3ffa6454b989815407f430e72/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:20:31.421219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130a2f3-851f-45f6-a7b9-21f19f284513 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:31.421933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130a2f3-851f-45f6-a7b9-21f19f284513 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:31.421933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130a2f3-851f-45f6-a7b9-21f19f284513 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:31.423112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 582/1154] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:31.428286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fb433-dc02-4ac9-a50a-053a79396b79 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:31.428684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fb433-dc02-4ac9-a50a-053a79396b79 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:31.428821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fb433-dc02-4ac9-a50a-053a79396b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:31.446421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-845fb433-dc02-4ac9-a50a-053a79396b79 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:31.447586 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-845fb433-dc02-4ac9-a50a-053a79396b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:31.511221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 573/1155] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:31.533794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130a2f3-851f-45f6-a7b9-21f19f284513 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:31.599165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:31.632729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130a2f3-851f-45f6-a7b9-21f19f284513 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:31.665474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:31.666084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:31.666358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:31.666622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:31.676236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:31.680853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:31.681985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:31.682977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 583/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:31.687976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fb433-dc02-4ac9-a50a-053a79396b79 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:31.781087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:31.784170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fb433-dc02-4ac9-a50a-053a79396b79 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:31.818336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:31.818929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:31.819207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:31.819477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:31.820131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:31.820433 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:31.821830 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:31.829578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 574/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:31.836080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:31.932276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602b45c-c93d-431b-b809-9f0dc41d4e17 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:31.957713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:31.966287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:31.966902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:31.967180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:31.967450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] PATH_INFO: `/v3/users/6f4efcc3ffa6454b989815407f430e72` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:31.968566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] RBAC: Authorizing `identity:get_user(user_id=6f4efcc3ffa6454b989815407f430e72)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:31.970050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:31.971610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 584/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:31 2022] GET /identity/v3/users/6f4efcc3ffa6454b989815407f430e72 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:31.977685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:32.074026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9126157-5819-4538-bbe2-bc2fa9a77c7c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:32.106546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:32.107139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:32.107417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:32.107688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:32.136929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:32.173399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:32.174847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:32.269767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 575/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:32.275503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:32.374406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb512efe-7f30-44a0-a68d-ad54e2acc12d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:32.406866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe3b03-7421-4628-99f1-55d53b666beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:32.407478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe3b03-7421-4628-99f1-55d53b666beb None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:32.407757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe3b03-7421-4628-99f1-55d53b666beb None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:32.408028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe3b03-7421-4628-99f1-55d53b666beb None admin] PATH_INFO: `/v3/projects/0a5c592379884eb492794b3d316145b0` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:32.409120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe3b03-7421-4628-99f1-55d53b666beb None admin] RBAC: Authorizing `identity:get_project(project_id=0a5c592379884eb492794b3d316145b0)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:32.410590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe3b03-7421-4628-99f1-55d53b666beb None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:32.412250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 585/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:32 2022] GET /identity/v3/projects/0a5c592379884eb492794b3d316145b0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:32.418378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:32.476540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:32.514527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3a7e6-760f-405a-84b4-1690307531a9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:32.547043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffa799-5929-4a5c-8dee-8fcbca220c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:32.547568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffa799-5929-4a5c-8dee-8fcbca220c49 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:32.547843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffa799-5929-4a5c-8dee-8fcbca220c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:32.548108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffa799-5929-4a5c-8dee-8fcbca220c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:32.548813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffa799-5929-4a5c-8dee-8fcbca220c49 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=0a5c592379884eb492794b3d316145b0, user.id=6f4efcc3ffa6454b989815407f430e72) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:32.549116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffa799-5929-4a5c-8dee-8fcbca220c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:32.550666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffa799-5929-4a5c-8dee-8fcbca220c49 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:32.556822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 576/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:32 2022] GET /identity/v3/role_assignments?user.id=6f4efcc3ffa6454b989815407f430e72&scope.project.id=0a5c592379884eb492794b3d316145b0&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:34.524245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88ab16-7e10-43de-81a2-be702b3d321e None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:34.524767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88ab16-7e10-43de-81a2-be702b3d321e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:34.524767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88ab16-7e10-43de-81a2-be702b3d321e None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:34.526159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 586/1162] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:34.531276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4926915-a658-4774-8f14-550bccb19f14 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:34.531691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4926915-a658-4774-8f14-550bccb19f14 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:34.531821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4926915-a658-4774-8f14-550bccb19f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:34.549886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d4926915-a658-4774-8f14-550bccb19f14 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:34.551049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d4926915-a658-4774-8f14-550bccb19f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:34.615366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 577/1163] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:34.637105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88ab16-7e10-43de-81a2-be702b3d321e None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:34.699086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:34.736976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88ab16-7e10-43de-81a2-be702b3d321e None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:34.769506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:34.770099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:34.770922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:34.771191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:34.834461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:34.835834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:34.892029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:34.928686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 587/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:34.934518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4926915-a658-4774-8f14-550bccb19f14 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:34.988071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:35.030483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4926915-a658-4774-8f14-550bccb19f14 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:35.062663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2552a-3be8-4c41-8556-2566ee515773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:35.063270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2552a-3be8-4c41-8556-2566ee515773 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:35.063544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2552a-3be8-4c41-8556-2566ee515773 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:35.063812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2552a-3be8-4c41-8556-2566ee515773 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:35.070755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2552a-3be8-4c41-8556-2566ee515773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:35.072174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2552a-3be8-4c41-8556-2566ee515773 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-dae2552a-3be8-4c41-8556-2566ee515773 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:35.083785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:35.085004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:35.085906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 578/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:20:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:20:35.090617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:35.187082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a5d65-1102-40c8-a45d-1c660634c11f None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:35.219469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df69bd-c9be-4075-a648-341535751ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:35.220057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df69bd-c9be-4075-a648-341535751ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:35.220333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df69bd-c9be-4075-a648-341535751ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:35.220599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df69bd-c9be-4075-a648-341535751ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:35.221228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df69bd-c9be-4075-a648-341535751ab6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:35.221557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df69bd-c9be-4075-a648-341535751ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:35.222932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df69bd-c9be-4075-a648-341535751ab6 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:35.230013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 588/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 10 12:20:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:37.178893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d384c3-099d-4647-b4eb-7adf8f4d3603 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:37.179435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d384c3-099d-4647-b4eb-7adf8f4d3603 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:37.179435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d384c3-099d-4647-b4eb-7adf8f4d3603 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:37.180788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 579/1167] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:37.185874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6c7cf-2489-4b40-8a67-c0232a991287 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:37.186284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6c7cf-2489-4b40-8a67-c0232a991287 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:37.186413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6c7cf-2489-4b40-8a67-c0232a991287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:37.204364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8ac6c7cf-2489-4b40-8a67-c0232a991287 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:37.205565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8ac6c7cf-2489-4b40-8a67-c0232a991287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:37.227998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:37.271226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 589/1168] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:37.293469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d384c3-099d-4647-b4eb-7adf8f4d3603 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:37.393105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d384c3-099d-4647-b4eb-7adf8f4d3603 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:37.408641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:37.426009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec62320-ce43-48bd-ae94-4b14ae450bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:37.427109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec62320-ce43-48bd-ae94-4b14ae450bfd None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:37.427384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec62320-ce43-48bd-ae94-4b14ae450bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:37.427649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec62320-ce43-48bd-ae94-4b14ae450bfd None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:37.428296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec62320-ce43-48bd-ae94-4b14ae450bfd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:37.429742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec62320-ce43-48bd-ae94-4b14ae450bfd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:37.445073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 580/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Wed Aug 10 12:20:37 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:20:39.432619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f1bb-9390-4075-9e6f-464e0b07b961 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:39.433172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f1bb-9390-4075-9e6f-464e0b07b961 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:39.433172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765f1bb-9390-4075-9e6f-464e0b07b961 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:39.434522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 590/1170] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:39.439547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f2d30-3ee4-405f-b990-b5f24a3ed463 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:39.439960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f2d30-3ee4-405f-b990-b5f24a3ed463 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:39.440094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f2d30-3ee4-405f-b990-b5f24a3ed463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:39.458617 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e55f2d30-3ee4-405f-b990-b5f24a3ed463 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:39.459779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e55f2d30-3ee4-405f-b990-b5f24a3ed463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:39.523129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 581/1171] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:39.545134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765f1bb-9390-4075-9e6f-464e0b07b961 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:39.642588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765f1bb-9390-4075-9e6f-464e0b07b961 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:39.674075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e01b3-50d3-486a-bdaf-c0eb060bddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:39.674678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e01b3-50d3-486a-bdaf-c0eb060bddab None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:39.674951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e01b3-50d3-486a-bdaf-c0eb060bddab None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:39.675218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e01b3-50d3-486a-bdaf-c0eb060bddab None admin] PATH_INFO: `/v3/domains/b5009d413ede47a7aaa5e82d590c4fa2` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:39.685610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e01b3-50d3-486a-bdaf-c0eb060bddab None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5009d413ede47a7aaa5e82d590c4fa2)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:39.686983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e01b3-50d3-486a-bdaf-c0eb060bddab None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:39.688544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 591/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 10 12:20:39 2022] GET /identity/v3/domains/b5009d413ede47a7aaa5e82d590c4fa2 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:39.694896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f2d30-3ee4-405f-b990-b5f24a3ed463 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:39.788823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f2d30-3ee4-405f-b990-b5f24a3ed463 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:39.795856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:39.823418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f977c-83b4-4b6d-ad98-e06bd7c27bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:39.824004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f977c-83b4-4b6d-ad98-e06bd7c27bde None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:39.824279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f977c-83b4-4b6d-ad98-e06bd7c27bde None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:39.824548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f977c-83b4-4b6d-ad98-e06bd7c27bde None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:39.825185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f977c-83b4-4b6d-ad98-e06bd7c27bde None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:39.826663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f977c-83b4-4b6d-ad98-e06bd7c27bde None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:39.851843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 582/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:20:39 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:20:41.836345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8814d-2018-4897-9bcb-62f134d6fd58 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:41.836862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8814d-2018-4897-9bcb-62f134d6fd58 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:41.836862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8814d-2018-4897-9bcb-62f134d6fd58 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:41.838449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 592/1174] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:41.843604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbfa4-e18e-4a4c-9cd8-6f66980ae07d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:41.844029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbfa4-e18e-4a4c-9cd8-6f66980ae07d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:41.844155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcbfa4-e18e-4a4c-9cd8-6f66980ae07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:41.862157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-25dcbfa4-e18e-4a4c-9cd8-6f66980ae07d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:41.863295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-25dcbfa4-e18e-4a4c-9cd8-6f66980ae07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:41.927657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 583/1175] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:41.951385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8814d-2018-4897-9bcb-62f134d6fd58 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:41.970663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:42.051764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8814d-2018-4897-9bcb-62f134d6fd58 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:42.085056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0471fab-829f-4463-8a93-0eccbd9b7dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:42.085777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0471fab-829f-4463-8a93-0eccbd9b7dac None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:42.086056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0471fab-829f-4463-8a93-0eccbd9b7dac None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:42.086324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0471fab-829f-4463-8a93-0eccbd9b7dac None admin] PATH_INFO: `/v3/domains/b5009d413ede47a7aaa5e82d590c4fa2` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:42.087415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0471fab-829f-4463-8a93-0eccbd9b7dac None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5009d413ede47a7aaa5e82d590c4fa2)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:42.088845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0471fab-829f-4463-8a93-0eccbd9b7dac None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:42.090429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 593/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 10 12:20:41 2022] GET /identity/v3/domains/b5009d413ede47a7aaa5e82d590c4fa2 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:42.096115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcbfa4-e18e-4a4c-9cd8-6f66980ae07d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:42.179247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:42.192042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcbfa4-e18e-4a4c-9cd8-6f66980ae07d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:42.225357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324ec5b-9571-4c5f-b288-aa72e67786a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:42.225951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324ec5b-9571-4c5f-b288-aa72e67786a5 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:42.226231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324ec5b-9571-4c5f-b288-aa72e67786a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:42.226506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324ec5b-9571-4c5f-b288-aa72e67786a5 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:42.227402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324ec5b-9571-4c5f-b288-aa72e67786a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:42.228821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324ec5b-9571-4c5f-b288-aa72e67786a5 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:42.249191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 584/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:20:42 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:20:44.256166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd161e3-0cb3-41be-a3c2-a906f79e0f67 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:44.256692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd161e3-0cb3-41be-a3c2-a906f79e0f67 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:44.256807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd161e3-0cb3-41be-a3c2-a906f79e0f67 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:44.258100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 594/1178] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:44.263206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8c35-ed67-4b8a-a38c-cf997445d9cc None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:44.263620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8c35-ed67-4b8a-a38c-cf997445d9cc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:44.263748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8c35-ed67-4b8a-a38c-cf997445d9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:44.282240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4a8a8c35-ed67-4b8a-a38c-cf997445d9cc None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:44.283457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4a8a8c35-ed67-4b8a-a38c-cf997445d9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:44.347423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 585/1179] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:44.370418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd161e3-0cb3-41be-a3c2-a906f79e0f67 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:44.454084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:44.469810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd161e3-0cb3-41be-a3c2-a906f79e0f67 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:44.502577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:44.503185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:44.503461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:44.503728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:44.513992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:44.518662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:44.519736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:44.526363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 595/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:44.527218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a8c35-ed67-4b8a-a38c-cf997445d9cc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:44.597210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:44.622517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a8c35-ed67-4b8a-a38c-cf997445d9cc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:44.655431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:44.656020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:44.656293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:44.656562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:44.657208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:44.657535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:44.658898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:44.666561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 586/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:44.672383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:44.768840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdbdd3-c340-40da-bd77-9a34cc446b33 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:44.802591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:44.803196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:44.803473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:44.803740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] PATH_INFO: `/v3/users/d477f51439f14aed8f07bd5d1d22dc5c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:44.816263 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] RBAC: Authorizing `identity:get_user(user_id=d477f51439f14aed8f07bd5d1d22dc5c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:44.818238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:44.819790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 596/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:44 2022] GET /identity/v3/users/d477f51439f14aed8f07bd5d1d22dc5c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:44.825379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:44.919997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734968b-a5d9-44d2-8d8a-19ddcc63cf4b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:44.952808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:44.953426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:44.953705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:44.953974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:44.957215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:45.018777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:45.020284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:45.113318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 587/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:45.119469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:45.123107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:45.215146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040cc00-5cf5-4349-b431-6a9b5fc07c40 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:45.247795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4976c56-ec5b-449c-9407-29fab57bb469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:45.248401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976c56-ec5b-449c-9407-29fab57bb469 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:45.248674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976c56-ec5b-449c-9407-29fab57bb469 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:45.248947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976c56-ec5b-449c-9407-29fab57bb469 None admin] PATH_INFO: `/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:45.250059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4976c56-ec5b-449c-9407-29fab57bb469 None admin] RBAC: Authorizing `identity:get_project(project_id=7ce5858e62cc4b8f9b7396e9df7abd66)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:45.251465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4976c56-ec5b-449c-9407-29fab57bb469 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:45.253033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 597/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:45 2022] GET /identity/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:45.259484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:45.292450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:45.354874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e064-76f3-4a77-87a1-748d144d5b12 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:45.387047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e53041-37ce-4f70-933c-b247415a3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:45.387630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53041-37ce-4f70-933c-b247415a3ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:45.387905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53041-37ce-4f70-933c-b247415a3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:45.388171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53041-37ce-4f70-933c-b247415a3ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:45.388905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e53041-37ce-4f70-933c-b247415a3ed2 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=7ce5858e62cc4b8f9b7396e9df7abd66, user.id=d477f51439f14aed8f07bd5d1d22dc5c) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:45.389209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e53041-37ce-4f70-933c-b247415a3ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:45.390628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e53041-37ce-4f70-933c-b247415a3ed2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:45.395746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 588/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:45 2022] GET /identity/v3/role_assignments?user.id=d477f51439f14aed8f07bd5d1d22dc5c&scope.project.id=7ce5858e62cc4b8f9b7396e9df7abd66&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:47.344839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468ed5-bd7e-4310-bf24-cb2cacd07d03 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:47.345385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468ed5-bd7e-4310-bf24-cb2cacd07d03 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:47.345502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468ed5-bd7e-4310-bf24-cb2cacd07d03 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:47.346760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 598/1186] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:47.352798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1467e-0481-434f-bb4e-98da92a4578e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:47.353180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1467e-0481-434f-bb4e-98da92a4578e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:47.353380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1467e-0481-434f-bb4e-98da92a4578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:47.370987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1db1467e-0481-434f-bb4e-98da92a4578e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:47.372167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1db1467e-0481-434f-bb4e-98da92a4578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:47.437324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 589/1187] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:47.459617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a468ed5-bd7e-4310-bf24-cb2cacd07d03 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:47.529149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:47.558426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a468ed5-bd7e-4310-bf24-cb2cacd07d03 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:47.591852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:47.592476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:47.592758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:47.593031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:47.604366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:47.609017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:47.610129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:20:47.611315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 599/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:47.616186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1467e-0481-434f-bb4e-98da92a4578e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:47.644800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:47.712663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1467e-0481-434f-bb4e-98da92a4578e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:47.745096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:47.745716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:47.745996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:47.746268 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:47.746924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:47.747229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:47.748612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:47.757045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 590/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:47.762811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:47.858454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e83152-ff2c-4cf7-b218-649b4de87fe7 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:47.886947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:47.892260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:47.892867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:47.893140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:47.893457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] PATH_INFO: `/v3/users/d477f51439f14aed8f07bd5d1d22dc5c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:47.894551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] RBAC: Authorizing `identity:get_user(user_id=d477f51439f14aed8f07bd5d1d22dc5c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:47.895995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:47.897568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 600/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:47 2022] GET /identity/v3/users/d477f51439f14aed8f07bd5d1d22dc5c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:47.903327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:47.991545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:48.000338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e001d-14a0-4c3d-a2d6-449e94406c8d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:48.032774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35539dd-bb59-4053-9e1d-83ddcdac5355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:48.033412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35539dd-bb59-4053-9e1d-83ddcdac5355 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:48.033693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35539dd-bb59-4053-9e1d-83ddcdac5355 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:48.033963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35539dd-bb59-4053-9e1d-83ddcdac5355 None admin] PATH_INFO: `/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:48.035058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35539dd-bb59-4053-9e1d-83ddcdac5355 None admin] RBAC: Authorizing `identity:get_project(project_id=7ce5858e62cc4b8f9b7396e9df7abd66)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:48.036500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35539dd-bb59-4053-9e1d-83ddcdac5355 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:48.038129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 591/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:47 2022] GET /identity/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:48.043854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:48.139478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104616e5-83ba-413a-8995-690e4366f82c None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:48.170968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4567e6e-1e23-420e-a2f0-a90a50b64ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:48.171610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4567e6e-1e23-420e-a2f0-a90a50b64ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:48.171888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4567e6e-1e23-420e-a2f0-a90a50b64ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:48.172161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4567e6e-1e23-420e-a2f0-a90a50b64ad5 None admin] PATH_INFO: `/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66/users/d477f51439f14aed8f07bd5d1d22dc5c/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:48.174057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4567e6e-1e23-420e-a2f0-a90a50b64ad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ce5858e62cc4b8f9b7396e9df7abd66, user_id=d477f51439f14aed8f07bd5d1d22dc5c, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:48.175480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4567e6e-1e23-420e-a2f0-a90a50b64ad5 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:48.188219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 601/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:20:48 2022] PUT /identity/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66/users/d477f51439f14aed8f07bd5d1d22dc5c/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:20:50.140915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a70e-301a-429a-8388-d7a48217524b None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:50.141475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a70e-301a-429a-8388-d7a48217524b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:50.141475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a70e-301a-429a-8388-d7a48217524b None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:50.142852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 592/1193] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:20:50.147982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7df4a-d573-4298-a9b2-fc1d1a60a242 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:50.148401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7df4a-d573-4298-a9b2-fc1d1a60a242 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:50.148534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7df4a-d573-4298-a9b2-fc1d1a60a242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:50.166116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-39a7df4a-d573-4298-a9b2-fc1d1a60a242 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:20:50.167283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-39a7df4a-d573-4298-a9b2-fc1d1a60a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:20:50.220911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:50.233116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 602/1194] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:20:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:20:50.256461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1a70e-301a-429a-8388-d7a48217524b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:50.357103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1a70e-301a-429a-8388-d7a48217524b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:50.388610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:50.389215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:50.389517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:50.389788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:50.400350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:20:50.404889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:20:50.406067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:20:50.407295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 593/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:20:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:20:50.412574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7df4a-d573-4298-a9b2-fc1d1a60a242 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:50.506911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7df4a-d573-4298-a9b2-fc1d1a60a242 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:50.539294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:50.539883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:50.540160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:50.540430 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:50.541088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:50.541423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:50.542780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:50.550430 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 603/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:20:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:50.555949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:50.578271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:50.654831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa263e15-2e67-4e09-83ed-7059ead6811c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:50.686642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:50.687255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:50.687534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:50.687808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] PATH_INFO: `/v3/users/d477f51439f14aed8f07bd5d1d22dc5c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:50.688910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] RBAC: Authorizing `identity:get_user(user_id=d477f51439f14aed8f07bd5d1d22dc5c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:50.690367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:50.692469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 594/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 10 12:20:50 2022] GET /identity/v3/users/d477f51439f14aed8f07bd5d1d22dc5c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:50.698166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:50.748667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:50.794448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df6a92-867c-411b-891e-e3ab48e68668 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:50.827606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:50.828196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:50.828472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:50.828741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:50.892946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:50.894410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:50.942422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:50.988077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 604/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:20:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:20:50.994331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:51.078445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:20:51.090088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d742b34-0e1a-4923-ad14-d7401c3ba009 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:51.122223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a489e72-551d-4772-b660-608b957d046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:51.122826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a489e72-551d-4772-b660-608b957d046e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:51.123098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a489e72-551d-4772-b660-608b957d046e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:51.123362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a489e72-551d-4772-b660-608b957d046e None admin] PATH_INFO: `/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:51.124436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a489e72-551d-4772-b660-608b957d046e None admin] RBAC: Authorizing `identity:get_project(project_id=7ce5858e62cc4b8f9b7396e9df7abd66)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:51.125890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a489e72-551d-4772-b660-608b957d046e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:51.127469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 595/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 10 12:20:50 2022] GET /identity/v3/projects/7ce5858e62cc4b8f9b7396e9df7abd66 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:20:51.133975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:20:51.227441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75a43-6b35-4ad5-9ad7-a3b37b69b6d4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:20:51.260117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5cb5f-083b-4312-a0e3-5934098eaea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:20:51.260700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5cb5f-083b-4312-a0e3-5934098eaea1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:20:51.260998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5cb5f-083b-4312-a0e3-5934098eaea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:20:51.261250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5cb5f-083b-4312-a0e3-5934098eaea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:20:51.261987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5cb5f-083b-4312-a0e3-5934098eaea1 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=7ce5858e62cc4b8f9b7396e9df7abd66, user.id=d477f51439f14aed8f07bd5d1d22dc5c) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:20:51.262292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5cb5f-083b-4312-a0e3-5934098eaea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:20:51.263702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5cb5f-083b-4312-a0e3-5934098eaea1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:20:51.269140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 605/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:20:51 2022] GET /identity/v3/role_assignments?user.id=d477f51439f14aed8f07bd5d1d22dc5c&scope.project.id=7ce5858e62cc4b8f9b7396e9df7abd66&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:01.118263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=22 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:01.266667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:45.631413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1dedc-3c55-4348-8f2a-9646d3826a75 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:45.632022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1dedc-3c55-4348-8f2a-9646d3826a75 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:45.632022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1dedc-3c55-4348-8f2a-9646d3826a75 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:45.633604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 596/1201] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:21:45.639389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34dd8-32ea-4f0e-92bc-eb7e6499ae87 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:45.639880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34dd8-32ea-4f0e-92bc-eb7e6499ae87 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:45.640073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34dd8-32ea-4f0e-92bc-eb7e6499ae87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:45.667294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-bfa34dd8-32ea-4f0e-92bc-eb7e6499ae87 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:21:45.668562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bfa34dd8-32ea-4f0e-92bc-eb7e6499ae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:21:45.757578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 606/1202] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:21:45.785616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1dedc-3c55-4348-8f2a-9646d3826a75 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:45.917976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1dedc-3c55-4348-8f2a-9646d3826a75 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:45.963698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:45.964373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:45.964748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:45.965095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:45.980630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:45.982218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:21:45.999439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:46.000785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:21:46.006703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:21:46.006703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:46.006703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:21:46.006703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 597/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:21:46.013328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa34dd8-32ea-4f0e-92bc-eb7e6499ae87 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:46.135882 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa34dd8-32ea-4f0e-92bc-eb7e6499ae87 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:46.181306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:46.190184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de723c0a-61af-47ff-b85b-aba69af75ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:46.191360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de723c0a-61af-47ff-b85b-aba69af75ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:46.191637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de723c0a-61af-47ff-b85b-aba69af75ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:46.191904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de723c0a-61af-47ff-b85b-aba69af75ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:46.192569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de723c0a-61af-47ff-b85b-aba69af75ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:46.192873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de723c0a-61af-47ff-b85b-aba69af75ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:46.194462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de723c0a-61af-47ff-b85b-aba69af75ce3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:46.208831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 607/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:46.215326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:46.308432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:46.379414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05291-2c63-4ee6-a439-95939be7f26d None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:46.421435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed762fa-c998-486b-966d-164af60bf36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:46.422105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed762fa-c998-486b-966d-164af60bf36b None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:46.422413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed762fa-c998-486b-966d-164af60bf36b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:46.422689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed762fa-c998-486b-966d-164af60bf36b None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:46.423337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed762fa-c998-486b-966d-164af60bf36b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:46.424778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed762fa-c998-486b-966d-164af60bf36b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:46.451970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 598/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 10 12:21:46 2022] POST /identity/v3/users => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:21:48.752483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38056-bc4a-4cac-a9ce-2f0f8a4e5cb5 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:48.752483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38056-bc4a-4cac-a9ce-2f0f8a4e5cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:48.752483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38056-bc4a-4cac-a9ce-2f0f8a4e5cb5 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:48.752483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 608/1206] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:21:48.758013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639ecd-dc74-4dc3-bdb0-af4fc081ac8a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:48.758420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639ecd-dc74-4dc3-bdb0-af4fc081ac8a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:48.758583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639ecd-dc74-4dc3-bdb0-af4fc081ac8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:48.777607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-23639ecd-dc74-4dc3-bdb0-af4fc081ac8a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:21:48.778812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-23639ecd-dc74-4dc3-bdb0-af4fc081ac8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:21:48.855718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 599/1207] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:21:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:21:48.878622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38056-bc4a-4cac-a9ce-2f0f8a4e5cb5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:48.993536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38056-bc4a-4cac-a9ce-2f0f8a4e5cb5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:49.037653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:49.038365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:49.038628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:49.038947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:49.050323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:21:49.055246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:49.056216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:21:49.057234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:49.061607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 609/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:21:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:49.062726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23639ecd-dc74-4dc3-bdb0-af4fc081ac8a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:49.109647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:49.184936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23639ecd-dc74-4dc3-bdb0-af4fc081ac8a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:49.234739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:49.235475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:49.235808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:49.236145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:49.236961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:49.237919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:49.239530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:49.249576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 600/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:21:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:21:49.255578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:49.321115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:49.385149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25066f-3096-4f10-b71b-5f7a31f515c4 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:49.429972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:49.430645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:49.430936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:49.431286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:49.442802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:49.444586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:21:49.463343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:49.464595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:21:49.465746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:21:49.465746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:49.465746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:49.469432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 610/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:21:49.470914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:49.602626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:49.606992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e5e95-62c5-4fac-b417-6360c83e596a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:49.650224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:49.651040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:49.651487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:49.651650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:49.652309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:49.652608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:49.654029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:49.668597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 601/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:49.674249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:49.799857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8730631-7685-400b-9c23-4a11359138ac None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:49.801836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:49.841292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:49.841947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:49.842239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:49.842485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:49.850900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:49.852610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:21:49.865189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:49.866348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:49.867307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 611/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:49.872384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:49.989553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdc023-f472-4e30-936c-340acc9c45d7 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:50.032383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:50.032994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:50.033291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:50.033575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:50.034329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:50.034647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:50.037092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:50.054636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 602/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 10 12:21:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:50.060740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:50.170873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa6e12-6e2d-4307-ad39-5da0e27a28ef None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:50.218512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:50.219304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:50.219595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:50.219867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:50.256234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:50.301293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:50.303347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:50.421718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 612/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:21:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:21:50.427428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:50.450786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:50.546925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f7671-fff6-425d-9bfc-55ffd8a39655 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:50.588726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:50.590490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:50.590769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:50.591043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:50.606758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:50.606758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:21:50.626904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:50.628423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:21:50.634317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:21:50.634317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:50.634317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:21:50.634317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 603/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:21:50.634552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:50.714068 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:50.763901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd4720-db59-4059-bec6-5607f0575617 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:50.803753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:50.803753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:50.804502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:50.804502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:50.804908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:50.805204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:50.806757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:50.824504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 613/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:50.825724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:50.902173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:50.937990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02597-00f6-440d-a35f-8b59d3cd77cc None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:50.972967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:50.973670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:50.973924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:50.974198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:50.982246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:50.983951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:21:50.997114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:50.998401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:21:50.999519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:21:51.005658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 604/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:21:51.007971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:51.127950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02804afa-e0d1-4835-a776-31cb680ef665 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:51.151561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:51.165882 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17004f98-0d2b-497f-b024-7c8a871f1ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:51.166542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17004f98-0d2b-497f-b024-7c8a871f1ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:51.166898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17004f98-0d2b-497f-b024-7c8a871f1ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:51.167081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17004f98-0d2b-497f-b024-7c8a871f1ab1 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:51.167821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17004f98-0d2b-497f-b024-7c8a871f1ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:51.168149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17004f98-0d2b-497f-b024-7c8a871f1ab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:51.169660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17004f98-0d2b-497f-b024-7c8a871f1ab1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:51.183591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 614/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:21:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:51.189799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:51.303380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d51ff-eb02-4611-8af5-e7912aa3b8a9 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:51.344955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:51.353673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cae39a-aed5-4e95-94b6-6cb17d486e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:51.354383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cae39a-aed5-4e95-94b6-6cb17d486e7e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:51.354713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cae39a-aed5-4e95-94b6-6cb17d486e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:51.355038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cae39a-aed5-4e95-94b6-6cb17d486e7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:51.355780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cae39a-aed5-4e95-94b6-6cb17d486e7e None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=18b7024ea0ae4deeba88ebd2f8d1f909) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:51.356083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cae39a-aed5-4e95-94b6-6cb17d486e7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:51.357711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cae39a-aed5-4e95-94b6-6cb17d486e7e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:51.364140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 605/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:21:51 2022] GET /identity/v3/role_assignments?user.id=18b7024ea0ae4deeba88ebd2f8d1f909&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:53.489896 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2dac-d7c9-4da9-a8e3-5ecc2aac25fa None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:53.490641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2dac-d7c9-4da9-a8e3-5ecc2aac25fa None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:53.490641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2dac-d7c9-4da9-a8e3-5ecc2aac25fa None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:53.491956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 615/1220] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:21:53.497602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8cfe9-f36a-4537-9cbd-02d4d9bc507b None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:53.498526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8cfe9-f36a-4537-9cbd-02d4d9bc507b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:53.498526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8cfe9-f36a-4537-9cbd-02d4d9bc507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:53.519845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e8a8cfe9-f36a-4537-9cbd-02d4d9bc507b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:21:53.521347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e8a8cfe9-f36a-4537-9cbd-02d4d9bc507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:21:53.600525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 606/1221] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:21:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:21:53.623999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d2dac-d7c9-4da9-a8e3-5ecc2aac25fa None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:53.730705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d2dac-d7c9-4da9-a8e3-5ecc2aac25fa None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:53.769303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:53.770220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:53.770624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:53.770883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:53.782252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:21:53.787372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:53.788600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:53.789733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 616/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:21:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:53.794560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8cfe9-f36a-4537-9cbd-02d4d9bc507b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:53.903040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8cfe9-f36a-4537-9cbd-02d4d9bc507b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:53.927583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:53.939741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:53.940493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:53.940768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:53.941036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:53.941751 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:53.942052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:53.943531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:53.952272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 607/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:21:53.959063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:54.004926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:54.066257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8aaf-028b-40eb-b00d-ea79bf9d30b1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:54.102179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:54.102810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:54.103110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:54.103495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:54.115868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:54.117525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:21:54.134968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:54.136159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:21:54.137242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:21:54.137242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:54.137242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:54.137242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 617/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:21:54.145039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:54.255263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef5058-2fdd-4f6c-813c-d24ce13ae8e2 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:54.288848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:54.289484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:54.289765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:54.290040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:54.290684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:54.290992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:54.292569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:54.307796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 608/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:54.314621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:54.423695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:54.431104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dda6c-05ed-41d0-8d69-be4c03b25271 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:54.467743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:54.468689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:54.468977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:54.469250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:54.477227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:54.478686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:21:54.490157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:54.491738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:54.492972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 618/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:54.497223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:54.549530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:54.615534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c9bc0-a8c7-4b22-b9e2-63e38cc8d71e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:54.654910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:54.655579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:54.656105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:54.656427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:54.657425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:54.657772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:54.659555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:54.680183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 609/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 10 12:21:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:54.685968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:54.803247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843249-130a-4b02-8c0c-e8ac97a8fa75 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:54.842088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:54.842773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:54.843129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:54.843444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:54.854842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:54.856414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:21:54.872217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:54.873441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:21:54.874525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:21:54.874525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:54.874525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:54.874525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 619/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:21:54.879975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:54.984089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfddf9-4c6c-40e3-9d60-2f692e53ae6a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:54.986416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:55.020546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:55.021133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:55.021491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:55.021709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:55.022348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:55.022650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:55.024009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:55.036170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 610/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:55.042255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:55.052694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:55.154939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58199136-7445-45ad-a541-b7fa268c66d7 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:55.193579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:55.194329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:55.194575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:55.194893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:55.202287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:55.203948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:21:55.215954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:55.217122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:55.218317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 620/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 12:21:55.224302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:55.337594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1202ce-728a-49cd-8230-817a7522c89a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:55.372291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba5f1c-38b7-4e17-9e6f-f5e822443d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:55.372888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba5f1c-38b7-4e17-9e6f-f5e822443d7e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:55.373164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba5f1c-38b7-4e17-9e6f-f5e822443d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:55.373461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba5f1c-38b7-4e17-9e6f-f5e822443d7e None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:55.374141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba5f1c-38b7-4e17-9e6f-f5e822443d7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:55.374444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba5f1c-38b7-4e17-9e6f-f5e822443d7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:55.375858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba5f1c-38b7-4e17-9e6f-f5e822443d7e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:55.387528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 611/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:55.394194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:55.465328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:55.500432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dc1cf-41a1-4ffd-b5e5-7721a6c50675 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:55.533965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc5e04-c90d-465b-bade-767635a35448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:55.535213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc5e04-c90d-465b-bade-767635a35448 None admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:55.535508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc5e04-c90d-465b-bade-767635a35448 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:55.535766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc5e04-c90d-465b-bade-767635a35448 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/18b7024ea0ae4deeba88ebd2f8d1f909/roles/8c3fb9b334554c2c831b5e08ab522856` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:55.551157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc5e04-c90d-465b-bade-767635a35448 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=18b7024ea0ae4deeba88ebd2f8d1f909, role_id=8c3fb9b334554c2c831b5e08ab522856)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:55.552697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc5e04-c90d-465b-bade-767635a35448 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:55.566406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 621/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:21:55 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/18b7024ea0ae4deeba88ebd2f8d1f909/roles/8c3fb9b334554c2c831b5e08ab522856 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:21:57.569697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a3071-9784-4f60-8dba-2d8c9596d4c6 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:57.570278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a3071-9784-4f60-8dba-2d8c9596d4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:57.570278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a3071-9784-4f60-8dba-2d8c9596d4c6 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:57.571581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 612/1233] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:21:57.576795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260198f-1dc2-4b0f-b433-2132665f23bf None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:57.577296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260198f-1dc2-4b0f-b433-2132665f23bf None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:57.577392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260198f-1dc2-4b0f-b433-2132665f23bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:57.595799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2260198f-1dc2-4b0f-b433-2132665f23bf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:21:57.597007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2260198f-1dc2-4b0f-b433-2132665f23bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:21:57.670015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 622/1234] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:21:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:21:57.694640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a3071-9784-4f60-8dba-2d8c9596d4c6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:57.719794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:57.798823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a3071-9784-4f60-8dba-2d8c9596d4c6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:57.834146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:57.834569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:57.834903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:57.835174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:57.845611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:21:57.851156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:57.852402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:21:57.853241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 613/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 10 12:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:57.858568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260198f-1dc2-4b0f-b433-2132665f23bf None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:57.910013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:57.966132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260198f-1dc2-4b0f-b433-2132665f23bf None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:58.001432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:58.002108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:58.002390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:58.002659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:58.003816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] RBAC: Adding query filter params (name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:58.004156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:58.005709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:58.013892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 623/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 10 12:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:21:58.020243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:58.115265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:58.132856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d31604-c44f-4482-89e1-49fb9fdbb035 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:58.169620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:58.170245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:58.170521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:58.170790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:58.183187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:58.184724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:21:58.202485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:58.203762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:21:58.204868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:21:58.204868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:58.204868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:21:58.204868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 614/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:58.210013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:58.324910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40988af-0842-4c83-bb33-307920ef644d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:58.335916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:58.363505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:58.364109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:58.364384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:58.364654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:58.365372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:58.365619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:58.366977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:58.379343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 624/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:58.385028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:58.490976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cea1aa-5ea3-4c79-8a00-3f19600dd959 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:58.526957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:58.527586 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:58.527864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:58.528133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:58.531054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:58.537578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:58.539365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:21:58.551732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:58.552878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:21:58.554024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 615/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:58.558952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:58.666715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d9a50-45e6-4896-94a5-0f9b2530f7f3 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:58.703760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:58.704484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:58.704824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:58.705162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:58.706181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:58.706573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:58.708183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:58.728487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 625/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 10 12:21:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:58.734443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:58.848741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77707bb3-c43f-47f9-bf65-243ae42434cb None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:58.885758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:58.886406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:58.886752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:58.887086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:58.960156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:58.967913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:58.969502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:59.077184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 616/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:21:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 12:21:59.083487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:59.114160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:59.197243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f292632-343b-4ad0-afce-72b30a6ea711 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:59.233020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:59.233709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:59.234012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:59.234285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:59.244244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:59.245786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:21:59.260698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:59.261949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:21:59.262835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:21:59.262835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:21:59.262835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:59.262835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 626/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:59.267766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:59.376371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:59.383471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924576be-508d-40d8-a108-4ddc1af89f98 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:59.423554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:59.424107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:59.424356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:59.424623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:59.425260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:59.425585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:59.426940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:59.438226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 617/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:21:59.444142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:59.533902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:21:59.558018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62cfe6-3c62-48cc-94a6-de15d2224baf None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:59.596188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:59.597065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:59.597196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:59.597487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:59.604689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:59.606192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:21:59.618109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:21:59.619230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:21:59.620115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 627/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:21:59.625234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:59.733067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdbe67-7caa-448b-9b7a-64f9837bd57a None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:59.772906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4ec7a-6164-4739-a56c-e3119f5e1987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:59.773727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4ec7a-6164-4739-a56c-e3119f5e1987 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:59.773864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4ec7a-6164-4739-a56c-e3119f5e1987 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:59.774145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4ec7a-6164-4739-a56c-e3119f5e1987 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:59.774837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4ec7a-6164-4739-a56c-e3119f5e1987 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:59.775141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4ec7a-6164-4739-a56c-e3119f5e1987 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:59.776619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4ec7a-6164-4739-a56c-e3119f5e1987 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:59.789547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 618/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:21:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:21:59.795409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:21:59.903707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef932ade-6cfc-4bd6-a533-be02b33ea21b None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:21:59.940246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2191a-08b4-43ef-86fd-d469435e25a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:21:59.941460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2191a-08b4-43ef-86fd-d469435e25a8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:21:59.941740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2191a-08b4-43ef-86fd-d469435e25a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:21:59.942016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2191a-08b4-43ef-86fd-d469435e25a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:21:59.942874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2191a-08b4-43ef-86fd-d469435e25a8 None admin] RBAC: Adding query filter params (role.id=8c3fb9b334554c2c831b5e08ab522856, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=18b7024ea0ae4deeba88ebd2f8d1f909) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:21:59.943279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2191a-08b4-43ef-86fd-d469435e25a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:21:59.944908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2191a-08b4-43ef-86fd-d469435e25a8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:21:59.951286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 628/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:21:59 2022] GET /identity/v3/role_assignments?user.id=18b7024ea0ae4deeba88ebd2f8d1f909&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=8c3fb9b334554c2c831b5e08ab522856 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:02.165738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e26aa-f73e-45c3-9f80-4dcc4c1dd83c None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:02.166365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e26aa-f73e-45c3-9f80-4dcc4c1dd83c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:02.166365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e26aa-f73e-45c3-9f80-4dcc4c1dd83c None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:02.168292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 619/1247] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:22:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:02.174870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163c70f-9f7d-4b77-b5b5-8e5687fad7a8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:02.175351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163c70f-9f7d-4b77-b5b5-8e5687fad7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:02.175442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163c70f-9f7d-4b77-b5b5-8e5687fad7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:02.182276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:02.195936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7163c70f-9f7d-4b77-b5b5-8e5687fad7a8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:02.197415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7163c70f-9f7d-4b77-b5b5-8e5687fad7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:02.281222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 629/1248] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 12:22:02.309540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e26aa-f73e-45c3-9f80-4dcc4c1dd83c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:02.332170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:02.433112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e26aa-f73e-45c3-9f80-4dcc4c1dd83c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:02.474868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:02.475595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:02.475875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:02.476144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:02.487896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:02.493406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:02.494535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:02.495702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 620/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:02.501786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163c70f-9f7d-4b77-b5b5-8e5687fad7a8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:02.612637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163c70f-9f7d-4b77-b5b5-8e5687fad7a8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:02.648419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:02.654478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:02.655184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:02.655437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:02.655760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:02.656518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:02.656831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:02.658369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:02.667224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 630/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:02.673104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:02.759025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:02.784427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c13d4-2bf1-463e-b710-d8ae559fd32e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:02.820832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:02.821647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:02.821919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:02.822203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:02.833161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:02.834637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:02.850210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:02.851336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:02.852621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:02.852621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:02.852621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:02.852621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 621/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:02.857929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:02.966386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8380c0-05be-434d-ba41-f0fd9da1db7d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:03.003852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:03.004472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:03.004753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:03.005022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:03.005751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:03.006095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:03.007592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:03.021326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 631/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:03.027474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:03.141015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67250ab4-dcd6-402a-b521-f8d9bb6e537b None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:03.170152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:03.179521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:03.180230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:03.180549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:03.180823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:03.188962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:03.190924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:22:03.202012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:03.203274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:03.204246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 622/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:03.209106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:03.274754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:03.328628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c72550-dd7b-4a8f-8bac-63d0935d9f78 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:03.369640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:03.370267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:03.370602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:03.370848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:03.371649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:03.371946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:03.373434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:03.393092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 632/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 10 12:22:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:22:03.399564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:03.511642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6dfd7-9ab7-4419-8bc3-1550d4b0b77e None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:03.554037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:03.555905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:03.555905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:03.555905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:03.621488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:03.634596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:03.636060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:03.744557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 623/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:22:03.750377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:03.855574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e7f3-f5df-491b-a89e-449a77816959 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:03.871710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:03.898500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:03.899128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:03.899404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:03.899675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:03.910372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:03.911900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:03.927253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:03.928289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:03.929631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:03.929631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:03.929631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:03.929631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 633/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:03.934464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:04.029124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:04.042013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e8879-2bc5-4aa6-874b-7f348b478f25 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:04.080841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:04.081474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:04.081847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:04.082073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:04.082702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:04.083031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:04.084459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:04.096106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 624/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:04.102734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:04.210325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7f9b6-c298-4370-9b13-f949a6f16f05 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:04.246038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:04.246662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:04.246959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:04.247231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:04.253807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:04.255233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:22:04.266203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:04.267217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:22:04.268374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:04.272311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 634/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:04.273443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:04.402534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958104bf-e0ee-4bbb-90cd-7babefaf980f None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:04.443559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78598a6-58cb-4082-a4da-21486d668d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:04.444161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78598a6-58cb-4082-a4da-21486d668d64 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:04.444440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78598a6-58cb-4082-a4da-21486d668d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:04.444710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78598a6-58cb-4082-a4da-21486d668d64 None admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:04.445425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78598a6-58cb-4082-a4da-21486d668d64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:04.445734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78598a6-58cb-4082-a4da-21486d668d64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:04.447151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78598a6-58cb-4082-a4da-21486d668d64 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:04.462498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 625/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:04.468608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:04.497467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:04.583695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22540c4a-a23b-4de0-871c-1464b1ee1333 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:04.620483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02bd4a-81dc-45ce-84ae-8249d26d4f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:04.621088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02bd4a-81dc-45ce-84ae-8249d26d4f75 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:04.621406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02bd4a-81dc-45ce-84ae-8249d26d4f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:04.621683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02bd4a-81dc-45ce-84ae-8249d26d4f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:04.622406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02bd4a-81dc-45ce-84ae-8249d26d4f75 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=18b7024ea0ae4deeba88ebd2f8d1f909) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:04.622718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02bd4a-81dc-45ce-84ae-8249d26d4f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:04.624159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02bd4a-81dc-45ce-84ae-8249d26d4f75 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:04.645375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 635/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:04 2022] GET /identity/v3/role_assignments?user.id=18b7024ea0ae4deeba88ebd2f8d1f909&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=2470da926a0342698ca115a00c86fea3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:06.633131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea56563-794c-4958-9eae-ae6f7f70c85e None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:06.633779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea56563-794c-4958-9eae-ae6f7f70c85e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:06.633779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea56563-794c-4958-9eae-ae6f7f70c85e None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:06.635080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 626/1261] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:06.640213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36a2a7-d80c-4f0b-90aa-6e134ce574b9 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:06.640622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36a2a7-d80c-4f0b-90aa-6e134ce574b9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:06.640756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36a2a7-d80c-4f0b-90aa-6e134ce574b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:06.658802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-0d36a2a7-d80c-4f0b-90aa-6e134ce574b9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:06.660511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0d36a2a7-d80c-4f0b-90aa-6e134ce574b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:06.723603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 636/1262] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:22:06.745674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea56563-794c-4958-9eae-ae6f7f70c85e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:06.756317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:06.845201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea56563-794c-4958-9eae-ae6f7f70c85e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:06.878942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:06.879492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:06.879769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:06.880041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:06.889786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:06.894967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:06.895956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:06.896876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 627/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:06.902007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36a2a7-d80c-4f0b-90aa-6e134ce574b9 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:06.920236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:06.997631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36a2a7-d80c-4f0b-90aa-6e134ce574b9 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:07.030955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:07.031572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.031845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.032109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.032760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:07.033053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:07.034482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:07.042070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 637/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:07.047954 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:07.113281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:07.145684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb09561-2488-4e90-bb7a-53fa92c13c2c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:07.177386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:07.177998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.178271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.178537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.188027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:07.189516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:07.203790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:07.204889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:07.206555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:07.206555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:07.206555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:07.206555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 628/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:07.211044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:07.288305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:07.308776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef499f-5d90-4a72-be76-377fc039b112 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:07.349629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:07.350336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.350606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.350931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.351945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:07.352296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:07.353976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:07.373960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 638/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:07.375157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:07.473219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ad20a-530e-41df-8025-aec01c989cfa None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:07.492744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:07.507343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:07.507954 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.508231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.508500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.515655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:07.517119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:22:07.527897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:07.533747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:07.534970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 629/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:07.535701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:07.630776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3f7ca-d51a-4169-adaa-e90e0843b0f0 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:07.662579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:07.663205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.663481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.663748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.664497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:07.664797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:07.666247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:07.675138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:07.683553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 639/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 10 12:22:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:07.689225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:07.785677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe4a45-bcdf-498d-bc75-0654327e8bc3 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:07.817475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:07.818075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.818350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.818619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.828518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:07.829986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:07.843319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:07.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:07.845746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:07.845746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:07.845746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:07.845746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 630/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:07.850492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:07.945947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8740-9527-4194-ba84-d2b5b3975670 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:07.978528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:07.979117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:07.979394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:07.979671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:07.980295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:07.980596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:07.981979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:07.993256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 640/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:07.998614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:08.028710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:08.095164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04148f4-718b-4c6f-b8f4-19ba0e63e9ad None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:08.128197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:08.128804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:08.129083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:08.129381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:08.135792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:08.137227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:22:08.148788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:08.149814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:08.150934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 631/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:08.155584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:08.212347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:08.253043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f100f66-8ea6-482e-af41-60aa93736b3d None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:08.285513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9601f6-5894-45e8-b2cf-109c46debe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:08.286098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9601f6-5894-45e8-b2cf-109c46debe38 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:08.286372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9601f6-5894-45e8-b2cf-109c46debe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:08.286639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9601f6-5894-45e8-b2cf-109c46debe38 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:08.287320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9601f6-5894-45e8-b2cf-109c46debe38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:08.287620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9601f6-5894-45e8-b2cf-109c46debe38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:08.289022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9601f6-5894-45e8-b2cf-109c46debe38 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:08.301992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 641/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:22:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:08.308232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:08.403376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:08.413209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422ed0-442f-4acf-96c6-78a4e08d81f4 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:08.445148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af4dad-1666-4f14-94b4-2ef431c74b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:08.445828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4dad-1666-4f14-94b4-2ef431c74b86 None admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:08.446112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4dad-1666-4f14-94b4-2ef431c74b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:08.446392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4dad-1666-4f14-94b4-2ef431c74b86 None admin] PATH_INFO: `/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/18b7024ea0ae4deeba88ebd2f8d1f909/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:08.448255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af4dad-1666-4f14-94b4-2ef431c74b86 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user_id=18b7024ea0ae4deeba88ebd2f8d1f909, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:08.449726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af4dad-1666-4f14-94b4-2ef431c74b86 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:08.462436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 632/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 10 12:22:08 2022] PUT /identity/v3/projects/ab58563f96aa4f1f8b29f3a3f19ca6b5/users/18b7024ea0ae4deeba88ebd2f8d1f909/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:22:10.452554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4ad1f-1b96-4a83-88b5-964000e95d09 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:10.453129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4ad1f-1b96-4a83-88b5-964000e95d09 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:10.453129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4ad1f-1b96-4a83-88b5-964000e95d09 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:10.454484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 642/1274] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:10.459647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd071c-00dd-4077-96cd-62018974f44f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:10.460026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd071c-00dd-4077-96cd-62018974f44f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:10.460204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd071c-00dd-4077-96cd-62018974f44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:10.478114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-dddd071c-00dd-4077-96cd-62018974f44f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:10.479295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-dddd071c-00dd-4077-96cd-62018974f44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:10.544248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 633/1275] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:22:10.567737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4ad1f-1b96-4a83-88b5-964000e95d09 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:10.667869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4ad1f-1b96-4a83-88b5-964000e95d09 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:10.684867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:10.701932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:10.702549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:10.702874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:10.703097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:10.712901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 12:22:10.717549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:10.718613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:10.719630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 643/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 10 12:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:10.724587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd071c-00dd-4077-96cd-62018974f44f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:10.803167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:10.820945 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd071c-00dd-4077-96cd-62018974f44f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:10.853674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:10.854266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:10.854543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:10.854812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:10.855459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:10.855759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:10.857112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:10.865486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 634/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 10 12:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:10.871133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:10.966819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d648-0eb8-4b21-8322-39a6d808fd31 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.000193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.000801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.001083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.001383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.011030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.012465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:11.026417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:11.027489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:11.028573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:11.028573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:11.028573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:11.028573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 644/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:11.032926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:11.129546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ea151-69ee-4e0c-98e5-50bd1a8826dd None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.162017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.162601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.162881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.163154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.163787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:11.164092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.165456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.176700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 635/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:11.182059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:11.212069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:11.279083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05260704-49d9-462a-bef3-91599ec2a82a None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.318736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.319432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.319779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.320085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.328371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.329834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 12:22:11.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:11.342224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:11.343318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 645/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:11.348541 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:11.352518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:11.445365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d440df-2a25-4902-a91f-04d34bdddabd None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.478088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.478677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.478953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.479223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.479936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:11.480241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.481697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.498498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 636/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 10 12:22:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:11.504665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:11.591179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:11.601445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbebf83-c5b1-4238-bd7c-807845879ef7 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.634129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.634723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.634998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.635265 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.700395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.701864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.786522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:11.796585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 646/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 10 12:22:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 12:22:11.802972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:11.861983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:11.899559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2989e68-50cd-43be-934b-f5fee31ec023 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:11.931786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:11.932390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:11.932664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:11.932932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:11.942442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:11.943878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 12:22:11.958110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:11.959195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 12:22:11.960095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 12:22:11.960095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 12:22:11.960095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:11.960095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 637/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:11.964678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:12.059177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69a19a-03e6-4a24-bdbd-f86a8bbe4908 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:12.091287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:12.091873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:12.092148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:12.092415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:12.093042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:12.093366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:12.094724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:12.106224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 647/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 10 12:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:12.111591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:12.209197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d067f-7864-49b1-95df-093312c2fb9c None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:12.222793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:12.242652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:12.243260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:12.243536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:12.243806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:12.250173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:12.251605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 12:22:12.262872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:12.263900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: ERROR keystone.server.flask.application Aug 10 12:22:12.265138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 638/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 10 12:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:12.270291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:12.322583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:12.371771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5749050-7a07-439e-b5c3-f7fa34c59ecc None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:12.404721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5a280-1503-4551-8da6-b2e2a50d4793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:12.405339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5a280-1503-4551-8da6-b2e2a50d4793 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:12.405614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5a280-1503-4551-8da6-b2e2a50d4793 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:12.405880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5a280-1503-4551-8da6-b2e2a50d4793 None admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:12.406555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5a280-1503-4551-8da6-b2e2a50d4793 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:12.406853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5a280-1503-4551-8da6-b2e2a50d4793 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:12.408258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5a280-1503-4551-8da6-b2e2a50d4793 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:12.419579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 648/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 10 12:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:12.425694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:12.523920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753383-e08c-44f5-b8e5-d814c00839de None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:12.555583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343837f-a0b3-4068-bab0-3f53f4af8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:12.556167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343837f-a0b3-4068-bab0-3f53f4af8637 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:12.556451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343837f-a0b3-4068-bab0-3f53f4af8637 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:12.556718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343837f-a0b3-4068-bab0-3f53f4af8637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:12.557462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343837f-a0b3-4068-bab0-3f53f4af8637 None admin] RBAC: Adding query filter params (role.id=2470da926a0342698ca115a00c86fea3, scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5, user.id=18b7024ea0ae4deeba88ebd2f8d1f909) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:12.557772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343837f-a0b3-4068-bab0-3f53f4af8637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:12.559184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343837f-a0b3-4068-bab0-3f53f4af8637 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:12.564885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 639/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 10 12:22:12 2022] GET /identity/v3/role_assignments?user.id=18b7024ea0ae4deeba88ebd2f8d1f909&scope.project.id=ab58563f96aa4f1f8b29f3a3f19ca6b5&role.id=2470da926a0342698ca115a00c86fea3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:14.536264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc172c0-0d41-4d12-85fe-eb1da24b739a None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:14.536800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc172c0-0d41-4d12-85fe-eb1da24b739a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:14.536800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc172c0-0d41-4d12-85fe-eb1da24b739a None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:14.538173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 649/1288] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:14.543219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99335008-cccb-4a0a-92e9-4c54d012f71e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:14.543605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99335008-cccb-4a0a-92e9-4c54d012f71e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:14.543834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99335008-cccb-4a0a-92e9-4c54d012f71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:14.561707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-99335008-cccb-4a0a-92e9-4c54d012f71e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:14.562872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-99335008-cccb-4a0a-92e9-4c54d012f71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:14.593030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:14.630930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 640/1289] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:22:14.652827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc172c0-0d41-4d12-85fe-eb1da24b739a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:14.750394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc172c0-0d41-4d12-85fe-eb1da24b739a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:14.761208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:14.784279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:14.784895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:14.785171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:14.785469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:14.786053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:14.787471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:14.796155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:14.797404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:14.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 650/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:22:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:14.803300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99335008-cccb-4a0a-92e9-4c54d012f71e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:14.899046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99335008-cccb-4a0a-92e9-4c54d012f71e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:14.931485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280269d2-f759-4964-9990-7b7ac58a8554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:14.932077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280269d2-f759-4964-9990-7b7ac58a8554 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:14.932357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280269d2-f759-4964-9990-7b7ac58a8554 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:14.932629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280269d2-f759-4964-9990-7b7ac58a8554 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:14.933255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280269d2-f759-4964-9990-7b7ac58a8554 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:14.933596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280269d2-f759-4964-9990-7b7ac58a8554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:14.934949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280269d2-f759-4964-9990-7b7ac58a8554 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:14.939615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 641/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 10 12:22:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:14.945901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:15.043120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19faf49-b391-473d-ad66-efb2638b29f8 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:15.076017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38260ba1-c39b-452e-9ef7-dda8696d9155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:15.076646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260ba1-c39b-452e-9ef7-dda8696d9155 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:15.076966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260ba1-c39b-452e-9ef7-dda8696d9155 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:15.077241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38260ba1-c39b-452e-9ef7-dda8696d9155 None admin] PATH_INFO: `/v3/services` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:15.077892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260ba1-c39b-452e-9ef7-dda8696d9155 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:15.078195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260ba1-c39b-452e-9ef7-dda8696d9155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:15.079558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38260ba1-c39b-452e-9ef7-dda8696d9155 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:15.083878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 651/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 10 12:22:14 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:17.024702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00883718-cd5b-4f23-8440-a63cd8aaeaa7 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.025220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00883718-cd5b-4f23-8440-a63cd8aaeaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.025220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00883718-cd5b-4f23-8440-a63cd8aaeaa7 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.026630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 642/1293] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:17.031744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe787cfa-96f5-484c-90ab-d9589a17bbcd None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.032138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe787cfa-96f5-484c-90ab-d9589a17bbcd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.032289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe787cfa-96f5-484c-90ab-d9589a17bbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.050951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fe787cfa-96f5-484c-90ab-d9589a17bbcd None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:17.052136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fe787cfa-96f5-484c-90ab-d9589a17bbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:17.080314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:17.117669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 652/1294] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:22:17.139098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00883718-cd5b-4f23-8440-a63cd8aaeaa7 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:17.170031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:17.237887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00883718-cd5b-4f23-8440-a63cd8aaeaa7 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:17.270116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232d4dd-70ff-454c-8e7b-1a45608d51f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:17.270701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232d4dd-70ff-454c-8e7b-1a45608d51f9 None admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:17.270979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232d4dd-70ff-454c-8e7b-1a45608d51f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:17.271252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232d4dd-70ff-454c-8e7b-1a45608d51f9 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:17.271887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232d4dd-70ff-454c-8e7b-1a45608d51f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:17.273253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232d4dd-70ff-454c-8e7b-1a45608d51f9 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:17.287238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 643/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 10 12:22:17 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:22:19.265053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7564f2-33f1-4b89-87a6-a700c96bafcf None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:19.265602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7564f2-33f1-4b89-87a6-a700c96bafcf None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:19.265602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7564f2-33f1-4b89-87a6-a700c96bafcf None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:19.266962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 653/1296] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:19.272502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689dbcb-1c53-401a-b1e9-6fbcfc1cc3fe None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:19.272863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689dbcb-1c53-401a-b1e9-6fbcfc1cc3fe None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:19.273056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689dbcb-1c53-401a-b1e9-6fbcfc1cc3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:19.290568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9689dbcb-1c53-401a-b1e9-6fbcfc1cc3fe None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:19.291737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9689dbcb-1c53-401a-b1e9-6fbcfc1cc3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:19.367836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 644/1297] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:22:19.390689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7564f2-33f1-4b89-87a6-a700c96bafcf None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:19.487423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7564f2-33f1-4b89-87a6-a700c96bafcf None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:19.519828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:19.520375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:19.520648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:19.520910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:19.521502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:19.522869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:19.531985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:19.533193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:19.534286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 654/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:22:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:19.538977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689dbcb-1c53-401a-b1e9-6fbcfc1cc3fe None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:19.557065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:19.637067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689dbcb-1c53-401a-b1e9-6fbcfc1cc3fe None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:19.668965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:19.669614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:19.669890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:19.670160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:19.670780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:19.671087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:19.672446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:19.677883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 645/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 10 12:22:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:19.683727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:19.714748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:19.782086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60716817-c192-43e8-91d2-09ea7e87a592 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:19.814876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2557e-72b2-4e1d-9697-8b88d95d1bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:19.815472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2557e-72b2-4e1d-9697-8b88d95d1bbc None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:19.815750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2557e-72b2-4e1d-9697-8b88d95d1bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:19.816020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2557e-72b2-4e1d-9697-8b88d95d1bbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:19.816713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2557e-72b2-4e1d-9697-8b88d95d1bbc None admin] RBAC: Adding query filter params (interface=public, service_id=1fe52fc28b414cb98d5875cb6fec70d2, region_id=RegionOne) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:19.817020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2557e-72b2-4e1d-9697-8b88d95d1bbc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:19.818412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2557e-72b2-4e1d-9697-8b88d95d1bbc None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:19.823310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 655/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 10 12:22:19 2022] GET /identity/v3/endpoints?service_id=1fe52fc28b414cb98d5875cb6fec70d2&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:19.828760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:19.908149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:19.927183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e75c8e-d2a0-489b-a39d-65d191224114 None admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:19.958979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3ccf2-ea78-4494-b4d9-081ada3af12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:19.959570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3ccf2-ea78-4494-b4d9-081ada3af12c None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:19.959851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3ccf2-ea78-4494-b4d9-081ada3af12c None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:19.960123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3ccf2-ea78-4494-b4d9-081ada3af12c None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:19.960718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3ccf2-ea78-4494-b4d9-081ada3af12c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:19.962106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3ccf2-ea78-4494-b4d9-081ada3af12c None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:19.968674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 646/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 10 12:22:19 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:22:21.934016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140d618-ad43-4ec7-94dd-1038bb69b099 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:21.934535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140d618-ad43-4ec7-94dd-1038bb69b099 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:21.934640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140d618-ad43-4ec7-94dd-1038bb69b099 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:21.935899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 656/1302] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:22:21.941597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff91af6-747f-42cb-9dd7-ada03e4e218e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:21.941960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff91af6-747f-42cb-9dd7-ada03e4e218e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:21.942149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff91af6-747f-42cb-9dd7-ada03e4e218e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:21.959813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3ff91af6-747f-42cb-9dd7-ada03e4e218e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:22:21.960999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3ff91af6-747f-42cb-9dd7-ada03e4e218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:22:22.025538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 647/1303] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:22:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 12:22:22.048581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140d618-ad43-4ec7-94dd-1038bb69b099 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:22.141671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:22.148747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140d618-ad43-4ec7-94dd-1038bb69b099 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:22.181902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:22.182515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:22.182786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:22.183050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:22.183618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:22.185009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 12:22:22.193290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 12:22:22.194496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application with Lock( Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self._enter() Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application return self.creator() Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application created_value = creator( Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: ERROR keystone.server.flask.application Aug 10 12:22:22.195339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 657/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 10 12:22:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 12:22:22.200358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff91af6-747f-42cb-9dd7-ada03e4e218e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:22.274197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:22.297752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff91af6-747f-42cb-9dd7-ada03e4e218e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:22.333379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03116e60-4583-47c7-bbc7-6e719ee5a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:22.334000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116e60-4583-47c7-bbc7-6e719ee5a326 None admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:22.334253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116e60-4583-47c7-bbc7-6e719ee5a326 None admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:22.334528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116e60-4583-47c7-bbc7-6e719ee5a326 None admin] PATH_INFO: `/v3/services` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:22.335181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03116e60-4583-47c7-bbc7-6e719ee5a326 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:22:22.335454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03116e60-4583-47c7-bbc7-6e719ee5a326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:22.336825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03116e60-4583-47c7-bbc7-6e719ee5a326 None admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:22.342147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 648/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 10 12:22:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:22:22.349026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:22:22.448814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ba8fd-650e-487d-90b2-6defff8bebe1 None admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:22:22.482038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29689f-4ac7-4671-a54f-60c8bb229c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:22:22.482633 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29689f-4ac7-4671-a54f-60c8bb229c1e None admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:22:22.482912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29689f-4ac7-4671-a54f-60c8bb229c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:22:22.483184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29689f-4ac7-4671-a54f-60c8bb229c1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:22:22.483831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29689f-4ac7-4671-a54f-60c8bb229c1e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:22:22.485201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29689f-4ac7-4671-a54f-60c8bb229c1e None admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:22:22.504463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 658/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:22:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:22:32.338988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=30 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:22:32.500272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.375071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daf95b-91e6-4855-8f17-39e9005cc4e8 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.376100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daf95b-91e6-4855-8f17-39e9005cc4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.376600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daf95b-91e6-4855-8f17-39e9005cc4e8 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.382718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 649/1307] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:24.383998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfebb7e-6b3d-44b6-9338-39351c217d98 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.384790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfebb7e-6b3d-44b6-9338-39351c217d98 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.385228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfebb7e-6b3d-44b6-9338-39351c217d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.471151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7bfebb7e-6b3d-44b6-9338-39351c217d98 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:24.472495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7bfebb7e-6b3d-44b6-9338-39351c217d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:24.591168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 659/1308] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:23:24.653748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c23c8-5197-4b4d-a678-ef29554fff6e None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.654168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c23c8-5197-4b4d-a678-ef29554fff6e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.654538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c23c8-5197-4b4d-a678-ef29554fff6e None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.657236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 650/1309] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 10 12:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:24.664397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72882-68f7-40c7-932e-bcbb35aefe07 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:24.665123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72882-68f7-40c7-932e-bcbb35aefe07 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:24.665493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72882-68f7-40c7-932e-bcbb35aefe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:24.819226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-93d72882-68f7-40c7-932e-bcbb35aefe07 None None] MFA Rules not processed for user `8c4588df9de242e8819de0476f75959c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:24.820511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-93d72882-68f7-40c7-932e-bcbb35aefe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:24.846945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:24.910502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 660/1310] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 10 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:23:24.919384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c23c8-5197-4b4d-a678-ef29554fff6e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:24.921107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-022c23c8-5197-4b4d-a678-ef29554fff6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:25.039049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c23c8-5197-4b4d-a678-ef29554fff6e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:25.114051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b95ed8-b42d-42e3-9dde-ba85ad76cca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:25.114660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b95ed8-b42d-42e3-9dde-ba85ad76cca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:25.114942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b95ed8-b42d-42e3-9dde-ba85ad76cca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:25.115213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b95ed8-b42d-42e3-9dde-ba85ad76cca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:25.116813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b2b95ed8-b42d-42e3-9dde-ba85ad76cca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:25.170768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:25.228868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b95ed8-b42d-42e3-9dde-ba85ad76cca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:25.230482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b95ed8-b42d-42e3-9dde-ba85ad76cca6 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:25.250082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 651/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:27.530446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa60d-99e4-440c-8509-61ac67e86951 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:27.530994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa60d-99e4-440c-8509-61ac67e86951 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:27.530994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa60d-99e4-440c-8509-61ac67e86951 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:27.532513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 661/1312] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:27.537432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9c481-8186-44f0-8da4-c4f9f6c85264 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:27.537817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9c481-8186-44f0-8da4-c4f9f6c85264 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:27.538034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9c481-8186-44f0-8da4-c4f9f6c85264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:27.625308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-38f9c481-8186-44f0-8da4-c4f9f6c85264 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:27.626534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-38f9c481-8186-44f0-8da4-c4f9f6c85264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:27.728265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 652/1313] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:27.768614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaa60d-99e4-440c-8509-61ac67e86951 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:27.770369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-edfaa60d-99e4-440c-8509-61ac67e86951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:27.896478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaa60d-99e4-440c-8509-61ac67e86951 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:27.981206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:27.985551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c61051-ba47-4a3a-bfae-32b3a19d1a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:27.986174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61051-ba47-4a3a-bfae-32b3a19d1a28 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:27.987115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61051-ba47-4a3a-bfae-32b3a19d1a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:27.987411 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c61051-ba47-4a3a-bfae-32b3a19d1a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:27.990050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-06c61051-ba47-4a3a-bfae-32b3a19d1a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:28.107140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c61051-ba47-4a3a-bfae-32b3a19d1a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:28.108706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c61051-ba47-4a3a-bfae-32b3a19d1a28 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:28.130114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 662/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:30.422203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c0b0-23b3-4941-bf54-5f886e431ea9 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.422724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c0b0-23b3-4941-bf54-5f886e431ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.422724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75c0b0-23b3-4941-bf54-5f886e431ea9 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.424721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 653/1315] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:30.429863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f97736-d860-4765-94b1-778b65f3c333 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.430280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f97736-d860-4765-94b1-778b65f3c333 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.430417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f97736-d860-4765-94b1-778b65f3c333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.495754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-42f97736-d860-4765-94b1-778b65f3c333 None None] MFA Rules not processed for user `6816207d6f6f4925a957b10d4c6e8140`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:30.496898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-42f97736-d860-4765-94b1-778b65f3c333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:30.546668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.590595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 663/1316] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:30.643507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722fe57-d49f-4c48-8a05-4608f76cc11b None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.643790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722fe57-d49f-4c48-8a05-4608f76cc11b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.644064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722fe57-d49f-4c48-8a05-4608f76cc11b None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.645461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 654/1317] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 10 12:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:30.651029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e233b0-462a-43b4-bbfe-de2229892cc5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.651312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e233b0-462a-43b4-bbfe-de2229892cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.651586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e233b0-462a-43b4-bbfe-de2229892cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.724902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-68e233b0-462a-43b4-bbfe-de2229892cc5 None None] MFA Rules not processed for user `8c4588df9de242e8819de0476f75959c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:30.726108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-68e233b0-462a-43b4-bbfe-de2229892cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:30.800603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 664/1318] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 10 12:23:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:30.811482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722fe57-d49f-4c48-8a05-4608f76cc11b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:30.813090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0722fe57-d49f-4c48-8a05-4608f76cc11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:30.822666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:30.916045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722fe57-d49f-4c48-8a05-4608f76cc11b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:30.985853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a808a1-804a-4b49-a002-17fc748745f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:30.986474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a808a1-804a-4b49-a002-17fc748745f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:30.986746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a808a1-804a-4b49-a002-17fc748745f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:30.987010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a808a1-804a-4b49-a002-17fc748745f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:30.988564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a7a808a1-804a-4b49-a002-17fc748745f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:31.047916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:31.093664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a808a1-804a-4b49-a002-17fc748745f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:31.095017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a808a1-804a-4b49-a002-17fc748745f9 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:31.117682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 655/1319] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:34.020468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9145190-1799-4e6c-80aa-28a9b7330672 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.021032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9145190-1799-4e6c-80aa-28a9b7330672 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.021032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9145190-1799-4e6c-80aa-28a9b7330672 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.022413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 665/1320] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:34.027537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52815c-a56a-4f47-b2c5-74c9ecfae92b None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.027936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52815c-a56a-4f47-b2c5-74c9ecfae92b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.028078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52815c-a56a-4f47-b2c5-74c9ecfae92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.091729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-0f52815c-a56a-4f47-b2c5-74c9ecfae92b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:34.092910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0f52815c-a56a-4f47-b2c5-74c9ecfae92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:34.188133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 656/1321] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:34.227273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9145190-1799-4e6c-80aa-28a9b7330672 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:34.228594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c9145190-1799-4e6c-80aa-28a9b7330672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:34.277978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:34.332199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9145190-1799-4e6c-80aa-28a9b7330672 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:34.403980 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437d1a-2a8f-472a-a64e-2f5c8e4454dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:34.404575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437d1a-2a8f-472a-a64e-2f5c8e4454dc service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:34.404850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437d1a-2a8f-472a-a64e-2f5c8e4454dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:34.405119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437d1a-2a8f-472a-a64e-2f5c8e4454dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:34.406789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7b437d1a-2a8f-472a-a64e-2f5c8e4454dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:34.499005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:34.512934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437d1a-2a8f-472a-a64e-2f5c8e4454dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:34.514395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437d1a-2a8f-472a-a64e-2f5c8e4454dc service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:34.531655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 666/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:37.198993 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97977af4-3250-4d65-aaab-a45c28ff1995 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.199516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97977af4-3250-4d65-aaab-a45c28ff1995 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.199516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97977af4-3250-4d65-aaab-a45c28ff1995 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.200898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 657/1323] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:37.206591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c5cd7-13a8-4938-bcaa-d300d65465f8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.207008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c5cd7-13a8-4938-bcaa-d300d65465f8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.207138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c5cd7-13a8-4938-bcaa-d300d65465f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.272747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-675c5cd7-13a8-4938-bcaa-d300d65465f8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:37.273986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-675c5cd7-13a8-4938-bcaa-d300d65465f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:37.378815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 667/1324] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:37.415697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97977af4-3250-4d65-aaab-a45c28ff1995 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:37.417259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-97977af4-3250-4d65-aaab-a45c28ff1995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:37.438300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.526169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97977af4-3250-4d65-aaab-a45c28ff1995 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:37.602014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a445d-931a-4d83-b1be-462bfcfaf40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:37.602646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a445d-931a-4d83-b1be-462bfcfaf40b service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:37.602920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a445d-931a-4d83-b1be-462bfcfaf40b service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:37.603184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a445d-931a-4d83-b1be-462bfcfaf40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:37.604762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-bc6a445d-931a-4d83-b1be-462bfcfaf40b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:37.674816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:37.713512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a445d-931a-4d83-b1be-462bfcfaf40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:37.714844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a445d-931a-4d83-b1be-462bfcfaf40b service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:37.733451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 658/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:39.768340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda703eb-2809-487b-a064-27904212a5dc None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.768842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda703eb-2809-487b-a064-27904212a5dc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.768842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda703eb-2809-487b-a064-27904212a5dc None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.770140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 668/1326] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:39.775382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde62f7-383d-4bd6-834f-f3ba3e3e62d6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:39.775833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde62f7-383d-4bd6-834f-f3ba3e3e62d6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:39.775944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde62f7-383d-4bd6-834f-f3ba3e3e62d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:39.841759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4dde62f7-383d-4bd6-834f-f3ba3e3e62d6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:39.842955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4dde62f7-383d-4bd6-834f-f3ba3e3e62d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:39.938693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 659/1327] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:39.974964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda703eb-2809-487b-a064-27904212a5dc None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:39.976783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-dda703eb-2809-487b-a064-27904212a5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:40.037434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.080678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda703eb-2809-487b-a064-27904212a5dc None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:40.156229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634d0c6-185f-478d-9912-29dfa3956acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:40.156848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634d0c6-185f-478d-9912-29dfa3956acd service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:40.157121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634d0c6-185f-478d-9912-29dfa3956acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:40.157418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634d0c6-185f-478d-9912-29dfa3956acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:40.159146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4634d0c6-185f-478d-9912-29dfa3956acd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:40.270002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:40.272417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634d0c6-185f-478d-9912-29dfa3956acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:40.274285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634d0c6-185f-478d-9912-29dfa3956acd service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:40.292545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 669/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:42.357417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1c26d-bbd7-4d65-9900-8f1e813e28fa None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.357956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1c26d-bbd7-4d65-9900-8f1e813e28fa None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.357956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1c26d-bbd7-4d65-9900-8f1e813e28fa None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.359328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 660/1329] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:42.364581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6b25b-dcef-40d7-a054-8a8440647ce3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.364993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6b25b-dcef-40d7-a054-8a8440647ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.365135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6b25b-dcef-40d7-a054-8a8440647ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.429961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-f3a6b25b-dcef-40d7-a054-8a8440647ce3 None None] MFA Rules not processed for user `6816207d6f6f4925a957b10d4c6e8140`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:42.431179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f3a6b25b-dcef-40d7-a054-8a8440647ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:42.529501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 670/1330] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:42.567844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1c26d-bbd7-4d65-9900-8f1e813e28fa None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:42.569482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-97e1c26d-bbd7-4d65-9900-8f1e813e28fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:42.603648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:42.675859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1c26d-bbd7-4d65-9900-8f1e813e28fa None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:42.749338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c389c8c-b727-44e3-aa50-e0bd697e3c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:42.749854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c389c8c-b727-44e3-aa50-e0bd697e3c02 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:42.750132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c389c8c-b727-44e3-aa50-e0bd697e3c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:42.750404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c389c8c-b727-44e3-aa50-e0bd697e3c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:42.752822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6c389c8c-b727-44e3-aa50-e0bd697e3c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:42.830370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:42.863038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c389c8c-b727-44e3-aa50-e0bd697e3c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:42.864488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c389c8c-b727-44e3-aa50-e0bd697e3c02 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:42.882549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 661/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:45.117023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612e087-5881-4552-ad30-c6194b2022a7 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.117556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612e087-5881-4552-ad30-c6194b2022a7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.117556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612e087-5881-4552-ad30-c6194b2022a7 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.118862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 671/1332] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:45.123985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1a0d-287e-48e9-8fff-971028b28cad None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.124413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1a0d-287e-48e9-8fff-971028b28cad None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.124559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1a0d-287e-48e9-8fff-971028b28cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.189792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-51fc1a0d-287e-48e9-8fff-971028b28cad None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:45.190989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-51fc1a0d-287e-48e9-8fff-971028b28cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:45.287447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 662/1333] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:45.323330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612e087-5881-4552-ad30-c6194b2022a7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:45.324856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1612e087-5881-4552-ad30-c6194b2022a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:45.393357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:45.426368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612e087-5881-4552-ad30-c6194b2022a7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:45.497482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680cfc9-1759-4767-9b30-cfa9a3ccb0e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:45.498076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cfc9-1759-4767-9b30-cfa9a3ccb0e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:45.498379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cfc9-1759-4767-9b30-cfa9a3ccb0e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:45.498650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cfc9-1759-4767-9b30-cfa9a3ccb0e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:45.500216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a680cfc9-1759-4767-9b30-cfa9a3ccb0e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:45.606553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680cfc9-1759-4767-9b30-cfa9a3ccb0e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:45.607995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680cfc9-1759-4767-9b30-cfa9a3ccb0e3 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:45.618197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:45.626563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 672/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:47.664920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb95d3c-e050-4cd2-b5db-06bd101fc549 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.665485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb95d3c-e050-4cd2-b5db-06bd101fc549 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.665590 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb95d3c-e050-4cd2-b5db-06bd101fc549 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.666968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 663/1335] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:47.672101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9fecf-862a-44fb-a18b-eade10c4f92f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:47.672524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9fecf-862a-44fb-a18b-eade10c4f92f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:47.672637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9fecf-862a-44fb-a18b-eade10c4f92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:47.739780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2fe9fecf-862a-44fb-a18b-eade10c4f92f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:47.740990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2fe9fecf-862a-44fb-a18b-eade10c4f92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:47.833611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 673/1336] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:47.874446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb95d3c-e050-4cd2-b5db-06bd101fc549 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:47.875960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8eb95d3c-e050-4cd2-b5db-06bd101fc549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:47.915850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:47.978462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb95d3c-e050-4cd2-b5db-06bd101fc549 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:48.049878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256a495-fbbc-4637-8d24-6d8dd800154e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:48.050473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a495-fbbc-4637-8d24-6d8dd800154e service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:48.050749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a495-fbbc-4637-8d24-6d8dd800154e service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:48.051016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a495-fbbc-4637-8d24-6d8dd800154e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:48.052582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2256a495-fbbc-4637-8d24-6d8dd800154e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:48.133703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:48.159879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256a495-fbbc-4637-8d24-6d8dd800154e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:48.161328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256a495-fbbc-4637-8d24-6d8dd800154e service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:48.178829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 664/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:50.515275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db9e8b-d708-464f-830f-4605f443fca1 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.515800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db9e8b-d708-464f-830f-4605f443fca1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.515800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db9e8b-d708-464f-830f-4605f443fca1 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.517084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 674/1338] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:50.522173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93387e-5f47-4240-8f47-ea7d17ce89f3 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.522570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93387e-5f47-4240-8f47-ea7d17ce89f3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.522720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93387e-5f47-4240-8f47-ea7d17ce89f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.587151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-7c93387e-5f47-4240-8f47-ea7d17ce89f3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:50.588348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7c93387e-5f47-4240-8f47-ea7d17ce89f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:50.684087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 665/1339] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:50.719506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db9e8b-d708-464f-830f-4605f443fca1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:50.721559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-09db9e8b-d708-464f-830f-4605f443fca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:50.795230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:50.823070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db9e8b-d708-464f-830f-4605f443fca1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:50.894746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219c929-4978-4939-aacc-b1c54bb15a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:50.895344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219c929-4978-4939-aacc-b1c54bb15a27 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:50.895621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219c929-4978-4939-aacc-b1c54bb15a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:50.895889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219c929-4978-4939-aacc-b1c54bb15a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:50.897483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4219c929-4978-4939-aacc-b1c54bb15a27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:51.003018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219c929-4978-4939-aacc-b1c54bb15a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:51.004460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219c929-4978-4939-aacc-b1c54bb15a27 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:51.022286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 675/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:54.856397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8912e-abee-45fa-865d-20a1fadb294a None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.856955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8912e-abee-45fa-865d-20a1fadb294a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.856955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8912e-abee-45fa-865d-20a1fadb294a None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.858546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 666/1341] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:54.863972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bec3c7-169b-4ecd-9fc8-6fd4a6e886a5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:54.864387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bec3c7-169b-4ecd-9fc8-6fd4a6e886a5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:54.864641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bec3c7-169b-4ecd-9fc8-6fd4a6e886a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:54.870165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:54.930782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b4bec3c7-169b-4ecd-9fc8-6fd4a6e886a5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:54.931986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b4bec3c7-169b-4ecd-9fc8-6fd4a6e886a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:55.028494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 676/1342] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:55.068074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8912e-abee-45fa-865d-20a1fadb294a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:55.069618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b9f8912e-abee-45fa-865d-20a1fadb294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:55.117919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.172872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8912e-abee-45fa-865d-20a1fadb294a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:55.245769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511cd3b-8b9b-43a5-8b78-385cbbdd0af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:55.246370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511cd3b-8b9b-43a5-8b78-385cbbdd0af5 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:55.246651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511cd3b-8b9b-43a5-8b78-385cbbdd0af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:55.246917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511cd3b-8b9b-43a5-8b78-385cbbdd0af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:55.248488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e511cd3b-8b9b-43a5-8b78-385cbbdd0af5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:55.341767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:55.358824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511cd3b-8b9b-43a5-8b78-385cbbdd0af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:55.360267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511cd3b-8b9b-43a5-8b78-385cbbdd0af5 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:55.380168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 667/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:23:58.034652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381b24a-eaed-49a4-9a37-8235b24b078d None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.035239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381b24a-eaed-49a4-9a37-8235b24b078d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.035239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381b24a-eaed-49a4-9a37-8235b24b078d None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.036560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 677/1344] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:23:58.042248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e154c4-fbea-4d46-98c1-93fe05d78181 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.042774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e154c4-fbea-4d46-98c1-93fe05d78181 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.042882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e154c4-fbea-4d46-98c1-93fe05d78181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.227473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e6e154c4-fbea-4d46-98c1-93fe05d78181 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:23:58.228687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e6e154c4-fbea-4d46-98c1-93fe05d78181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:58.327591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 668/1345] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:23:58.365499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381b24a-eaed-49a4-9a37-8235b24b078d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:23:58.367084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8381b24a-eaed-49a4-9a37-8235b24b078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:58.452901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:23:58.471474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381b24a-eaed-49a4-9a37-8235b24b078d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:23:58.544487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9c20-b579-4488-a4f3-af2ea00b5e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:23:58.545103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9c20-b579-4488-a4f3-af2ea00b5e68 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:23:58.545412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9c20-b579-4488-a4f3-af2ea00b5e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:23:58.545682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9c20-b579-4488-a4f3-af2ea00b5e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:23:58.547263 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0a8b9c20-b579-4488-a4f3-af2ea00b5e68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:23:58.659475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b9c20-b579-4488-a4f3-af2ea00b5e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:23:58.660942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b9c20-b579-4488-a4f3-af2ea00b5e68 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:23:58.679392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 678/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:02.603422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f134bb4-5861-4578-973a-2b479ff1b71f None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.603866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f134bb4-5861-4578-973a-2b479ff1b71f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.603974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f134bb4-5861-4578-973a-2b479ff1b71f None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.605487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 669/1347] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:02.610572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95961ec1-59bb-4016-8448-d407d2975c09 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:02.611600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95961ec1-59bb-4016-8448-d407d2975c09 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:02.611796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95961ec1-59bb-4016-8448-d407d2975c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:02.627503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:02.680251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-95961ec1-59bb-4016-8448-d407d2975c09 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:02.681483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-95961ec1-59bb-4016-8448-d407d2975c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:02.807051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 679/1348] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:02.844675 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f134bb4-5861-4578-973a-2b479ff1b71f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:02.846716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4f134bb4-5861-4578-973a-2b479ff1b71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:02.910083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:02.959989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f134bb4-5861-4578-973a-2b479ff1b71f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:03.037797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c838f-bd2a-4372-8eba-271de1b073e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:03.038416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c838f-bd2a-4372-8eba-271de1b073e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:03.038692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c838f-bd2a-4372-8eba-271de1b073e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:03.038962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c838f-bd2a-4372-8eba-271de1b073e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:03.040570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0a3c838f-bd2a-4372-8eba-271de1b073e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:03.152072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:03.155821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c838f-bd2a-4372-8eba-271de1b073e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:03.157394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c838f-bd2a-4372-8eba-271de1b073e4 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:03.177027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 670/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:08.008250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f787a63-41f0-453a-aed0-4172212f5385 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.008769 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f787a63-41f0-453a-aed0-4172212f5385 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.008851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f787a63-41f0-453a-aed0-4172212f5385 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.012981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 680/1350] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:24:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:08.022861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad93f5-48f9-482a-be57-0728827d12e1 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:08.023267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad93f5-48f9-482a-be57-0728827d12e1 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:08.023477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad93f5-48f9-482a-be57-0728827d12e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:08.137596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b7ad93f5-48f9-482a-be57-0728827d12e1 None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:08.139052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b7ad93f5-48f9-482a-be57-0728827d12e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:08.265578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 671/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:12.334767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71537-5b5a-45e6-b4ac-522839025c2b None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.335301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71537-5b5a-45e6-b4ac-522839025c2b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.335301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71537-5b5a-45e6-b4ac-522839025c2b None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.336834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 681/1352] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:12.341695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622457a-c08d-4bac-a90e-5d19383915a7 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:12.342124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622457a-c08d-4bac-a90e-5d19383915a7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:12.342469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622457a-c08d-4bac-a90e-5d19383915a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:12.406703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:12.433010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-c622457a-c08d-4bac-a90e-5d19383915a7 None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:12.434298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c622457a-c08d-4bac-a90e-5d19383915a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:12.546921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 672/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:24:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:12.805155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=59 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:13.964874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8920c-4da8-4929-ab77-d8298ccc31d9 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.965580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8920c-4da8-4929-ab77-d8298ccc31d9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.965580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8920c-4da8-4929-ab77-d8298ccc31d9 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:13.966802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 682/1354] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:13.971618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c562bf-568b-479a-a83d-4b3a87e90e2c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:13.972010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c562bf-568b-479a-a83d-4b3a87e90e2c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:13.972283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c562bf-568b-479a-a83d-4b3a87e90e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:14.049235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-46c562bf-568b-479a-a83d-4b3a87e90e2c None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:14.050497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-46c562bf-568b-479a-a83d-4b3a87e90e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:14.093556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:14.157317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 673/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:17.479075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04e82-09fb-4b56-a8ab-8e5dba564539 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.479591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04e82-09fb-4b56-a8ab-8e5dba564539 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.479591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04e82-09fb-4b56-a8ab-8e5dba564539 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.483369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 683/1356] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:17.487934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30903dba-d16a-411f-924d-7a18f91bbcaf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.488738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30903dba-d16a-411f-924d-7a18f91bbcaf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.489176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30903dba-d16a-411f-924d-7a18f91bbcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.585007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-30903dba-d16a-411f-924d-7a18f91bbcaf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:17.586210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-30903dba-d16a-411f-924d-7a18f91bbcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:17.693840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 674/1357] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:17.750416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d3960-efba-41b2-9f93-66810deac38e None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.750967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d3960-efba-41b2-9f93-66810deac38e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.751618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d3960-efba-41b2-9f93-66810deac38e None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.753860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 684/1358] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 10 12:24:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:17.760167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ea6d7-c5c5-4cd0-9367-c1a75a958888 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:17.760451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ea6d7-c5c5-4cd0-9367-c1a75a958888 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:17.760724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ea6d7-c5c5-4cd0-9367-c1a75a958888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:17.778679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:17.844686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e12ea6d7-c5c5-4cd0-9367-c1a75a958888 None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:17.846481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e12ea6d7-c5c5-4cd0-9367-c1a75a958888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:17.975899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 675/1359] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 10 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:24:17.984215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d3960-efba-41b2-9f93-66810deac38e None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:17.985793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-451d3960-efba-41b2-9f93-66810deac38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:18.141121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d3960-efba-41b2-9f93-66810deac38e None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:18.223038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:18.269799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec15105-1f51-47b8-8d31-f200f7d24bbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:18.270408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec15105-1f51-47b8-8d31-f200f7d24bbd service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:18.270679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec15105-1f51-47b8-8d31-f200f7d24bbd service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:18.270943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec15105-1f51-47b8-8d31-f200f7d24bbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:18.272519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1ec15105-1f51-47b8-8d31-f200f7d24bbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:18.370309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec15105-1f51-47b8-8d31-f200f7d24bbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:18.371905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec15105-1f51-47b8-8d31-f200f7d24bbd service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:18.390760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 685/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:20.616791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abac83c-8680-4926-b20e-3fe2068d45d3 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.617412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abac83c-8680-4926-b20e-3fe2068d45d3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.617412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abac83c-8680-4926-b20e-3fe2068d45d3 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.618976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 676/1361] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:20.624144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ab721-2edd-46cc-82a1-1170ce5f96e3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:20.624656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ab721-2edd-46cc-82a1-1170ce5f96e3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:20.624813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ab721-2edd-46cc-82a1-1170ce5f96e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:20.693872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ef6ab721-2edd-46cc-82a1-1170ce5f96e3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:20.695075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ef6ab721-2edd-46cc-82a1-1170ce5f96e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:20.741139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:20.799681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 686/1362] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:20.844426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abac83c-8680-4926-b20e-3fe2068d45d3 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:20.846786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8abac83c-8680-4926-b20e-3fe2068d45d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:20.934295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:21.031379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abac83c-8680-4926-b20e-3fe2068d45d3 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:21.188697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11efa5c-c80a-4a10-9c22-738fa8f8755d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:21.189340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11efa5c-c80a-4a10-9c22-738fa8f8755d service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:21.189723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11efa5c-c80a-4a10-9c22-738fa8f8755d service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:21.189950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11efa5c-c80a-4a10-9c22-738fa8f8755d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:21.191482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e11efa5c-c80a-4a10-9c22-738fa8f8755d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:21.215319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:21.286755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11efa5c-c80a-4a10-9c22-738fa8f8755d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:21.288398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11efa5c-c80a-4a10-9c22-738fa8f8755d service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:21.307350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 677/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:23.675104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56995-355d-4fcd-a892-bcf2dcefc6c1 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.675626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56995-355d-4fcd-a892-bcf2dcefc6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.675890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56995-355d-4fcd-a892-bcf2dcefc6c1 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.677758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 687/1364] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:23.683226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695bac-108e-422a-b350-62aa7bddc7ba None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.683691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695bac-108e-422a-b350-62aa7bddc7ba None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.683915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695bac-108e-422a-b350-62aa7bddc7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.760557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-74695bac-108e-422a-b350-62aa7bddc7ba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:23.761878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-74695bac-108e-422a-b350-62aa7bddc7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:23.873707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:23.881308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 678/1365] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:23.923312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7544f-68cd-4bbf-a514-63b2abc88cf4 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.924108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7544f-68cd-4bbf-a514-63b2abc88cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.924583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7544f-68cd-4bbf-a514-63b2abc88cf4 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:23.925996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 688/1366] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 10 12:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:23.931391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699de5-7b33-4c03-a88b-9d57a703ff79 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:23.931813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699de5-7b33-4c03-a88b-9d57a703ff79 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:23.932200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b699de5-7b33-4c03-a88b-9d57a703ff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.014134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1b699de5-7b33-4c03-a88b-9d57a703ff79 None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:24.015498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1b699de5-7b33-4c03-a88b-9d57a703ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:24.139552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 679/1367] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 10 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:24.148374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7544f-68cd-4bbf-a514-63b2abc88cf4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:24.150024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-23d7544f-68cd-4bbf-a514-63b2abc88cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:24.318910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7544f-68cd-4bbf-a514-63b2abc88cf4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:24.341076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:24.450575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f5427-ea29-4421-bfe6-9a0387bccbf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:24.451188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f5427-ea29-4421-bfe6-9a0387bccbf6 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:24.451469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f5427-ea29-4421-bfe6-9a0387bccbf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:24.451742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f5427-ea29-4421-bfe6-9a0387bccbf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:24.453361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f73f5427-ea29-4421-bfe6-9a0387bccbf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:24.551717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f5427-ea29-4421-bfe6-9a0387bccbf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:24.553368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f5427-ea29-4421-bfe6-9a0387bccbf6 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:24.573530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 689/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:25.519058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9b1b-1807-4cd9-b90f-7515a2fbd249 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.519669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9b1b-1807-4cd9-b90f-7515a2fbd249 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.519669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9b1b-1807-4cd9-b90f-7515a2fbd249 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.521499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 680/1369] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:25.526576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee0a7-287a-41d2-984b-ff6ea05b3884 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:25.527038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee0a7-287a-41d2-984b-ff6ea05b3884 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:25.527120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ee0a7-287a-41d2-984b-ff6ea05b3884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:25.559316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:25.595456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-cc7ee0a7-287a-41d2-984b-ff6ea05b3884 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:25.596669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-cc7ee0a7-287a-41d2-984b-ff6ea05b3884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:25.695877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 690/1370] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:26.037724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6334fdb-3db3-4cec-92e1-2564f9fe410a None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:26.037724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6334fdb-3db3-4cec-92e1-2564f9fe410a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:26.037724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6334fdb-3db3-4cec-92e1-2564f9fe410a None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:26.037724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 681/1371] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 10 12:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:26.042150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1a200-6271-453a-8d5c-7d85e524a858 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:26.042437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1a200-6271-453a-8d5c-7d85e524a858 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:26.042708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1a200-6271-453a-8d5c-7d85e524a858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:26.124030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-cec1a200-6271-453a-8d5c-7d85e524a858 None None] MFA Rules not processed for user `89d07f733a6c44858ba396c933e370e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:26.125231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-cec1a200-6271-453a-8d5c-7d85e524a858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:26.174407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:26.206008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 691/1372] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 10 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:26.213651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6334fdb-3db3-4cec-92e1-2564f9fe410a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:26.215248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e6334fdb-3db3-4cec-92e1-2564f9fe410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:26.280733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:26.347009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6334fdb-3db3-4cec-92e1-2564f9fe410a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:26.427368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658698-150f-44dc-b4f4-3082bd850e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:26.427980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658698-150f-44dc-b4f4-3082bd850e1a service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:26.428256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658698-150f-44dc-b4f4-3082bd850e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:26.428524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658698-150f-44dc-b4f4-3082bd850e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:26.430127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-44658698-150f-44dc-b4f4-3082bd850e1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:26.534535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:26.552480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658698-150f-44dc-b4f4-3082bd850e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:26.553971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658698-150f-44dc-b4f4-3082bd850e1a service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:26.586143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 682/1373] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:26.710116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade24e-4691-4939-ab1b-80b48d3b3171 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:26.710666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade24e-4691-4939-ab1b-80b48d3b3171 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:26.710666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade24e-4691-4939-ab1b-80b48d3b3171 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:26.713660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 692/1374] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:26.717252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06fa21-d9df-4ec3-8fa0-051a0ef33622 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:26.717561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06fa21-d9df-4ec3-8fa0-051a0ef33622 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:26.717835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06fa21-d9df-4ec3-8fa0-051a0ef33622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:26.787622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4e06fa21-d9df-4ec3-8fa0-051a0ef33622 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:26.789066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4e06fa21-d9df-4ec3-8fa0-051a0ef33622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:26.895607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 683/1375] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:26.940508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade24e-4691-4939-ab1b-80b48d3b3171 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:26.942455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2bade24e-4691-4939-ab1b-80b48d3b3171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:27.091161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade24e-4691-4939-ab1b-80b48d3b3171 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:27.143576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:27.206769 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e92e-da5d-42e8-8424-74f8b4a49f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:27.207386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e92e-da5d-42e8-8424-74f8b4a49f07 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:27.207653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e92e-da5d-42e8-8424-74f8b4a49f07 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:27.207923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e92e-da5d-42e8-8424-74f8b4a49f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:27.209541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bd16e92e-da5d-42e8-8424-74f8b4a49f07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:27.302176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16e92e-da5d-42e8-8424-74f8b4a49f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:27.303836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16e92e-da5d-42e8-8424-74f8b4a49f07 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:27.322908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 693/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:29.349814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df5823-c07b-45fb-b35e-da7bc410d5be None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.350301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df5823-c07b-45fb-b35e-da7bc410d5be None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.350301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df5823-c07b-45fb-b35e-da7bc410d5be None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.351924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 684/1377] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:29.357182 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815658-defd-416f-90aa-6f4d052d319f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.357614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815658-defd-416f-90aa-6f4d052d319f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.357773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815658-defd-416f-90aa-6f4d052d319f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.430366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-65815658-defd-416f-90aa-6f4d052d319f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:29.431572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-65815658-defd-416f-90aa-6f4d052d319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:29.438250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:29.547407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 694/1378] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:29.604991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df5823-c07b-45fb-b35e-da7bc410d5be None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:29.606772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e2df5823-c07b-45fb-b35e-da7bc410d5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:29.669683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:29.755844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df5823-c07b-45fb-b35e-da7bc410d5be None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:29.875191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a54a3-11c4-41c3-85f7-02ef9d474d50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:29.875932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a54a3-11c4-41c3-85f7-02ef9d474d50 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:29.876306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a54a3-11c4-41c3-85f7-02ef9d474d50 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:29.876676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a54a3-11c4-41c3-85f7-02ef9d474d50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:29.878727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-149a54a3-11c4-41c3-85f7-02ef9d474d50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:29.898798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:29.992378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a54a3-11c4-41c3-85f7-02ef9d474d50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:29.994029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a54a3-11c4-41c3-85f7-02ef9d474d50 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:30.033495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 685/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:32.125757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da11b7-9092-4f6a-803c-f07748971457 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.126903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da11b7-9092-4f6a-803c-f07748971457 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.126903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da11b7-9092-4f6a-803c-f07748971457 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.128466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 695/1380] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:32.134146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cda378-c271-4e90-ae91-185a6c01443d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.134683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cda378-c271-4e90-ae91-185a6c01443d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.134683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cda378-c271-4e90-ae91-185a6c01443d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.208657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-69cda378-c271-4e90-ae91-185a6c01443d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:32.209802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-69cda378-c271-4e90-ae91-185a6c01443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:32.305667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:32.312796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 686/1381] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:32.357111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da11b7-9092-4f6a-803c-f07748971457 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:32.358668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-38da11b7-9092-4f6a-803c-f07748971457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:32.505424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:32.516635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da11b7-9092-4f6a-803c-f07748971457 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:32.641044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c0b43-1989-444d-bc7b-e6472c2173de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:32.641703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0b43-1989-444d-bc7b-e6472c2173de service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:32.641977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0b43-1989-444d-bc7b-e6472c2173de service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:32.642243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0b43-1989-444d-bc7b-e6472c2173de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:32.643794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-575c0b43-1989-444d-bc7b-e6472c2173de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:32.742593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c0b43-1989-444d-bc7b-e6472c2173de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:32.744106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c0b43-1989-444d-bc7b-e6472c2173de service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:32.755270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:32.764505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 696/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:34.812913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aa650-ff30-4b76-b62d-ab7b9b6f0b99 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.814000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aa650-ff30-4b76-b62d-ab7b9b6f0b99 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.814262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aa650-ff30-4b76-b62d-ab7b9b6f0b99 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.816092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 687/1383] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:34.821310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0145-8f4a-46c4-b537-d61e1b4448b9 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:34.821721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0145-8f4a-46c4-b537-d61e1b4448b9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:34.821875 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0145-8f4a-46c4-b537-d61e1b4448b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:34.889871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8f3f0145-8f4a-46c4-b537-d61e1b4448b9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:34.890999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8f3f0145-8f4a-46c4-b537-d61e1b4448b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:34.991079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 697/1384] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:35.034031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aa650-ff30-4b76-b62d-ab7b9b6f0b99 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:35.036006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-447aa650-ff30-4b76-b62d-ab7b9b6f0b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:35.205995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aa650-ff30-4b76-b62d-ab7b9b6f0b99 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:35.280619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:35.330897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef85e8a-b261-4f9c-9027-b74fe9665614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:35.331499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef85e8a-b261-4f9c-9027-b74fe9665614 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:35.331777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef85e8a-b261-4f9c-9027-b74fe9665614 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:35.332039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef85e8a-b261-4f9c-9027-b74fe9665614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:35.333652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1ef85e8a-b261-4f9c-9027-b74fe9665614 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:35.434415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef85e8a-b261-4f9c-9027-b74fe9665614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:35.436600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef85e8a-b261-4f9c-9027-b74fe9665614 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:35.460422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 688/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:37.847472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9a4aa-0648-4a2c-91a8-9220438a0795 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:37.847994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9a4aa-0648-4a2c-91a8-9220438a0795 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:37.847994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9a4aa-0648-4a2c-91a8-9220438a0795 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:37.849326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 698/1386] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:37.854846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742dfb-5727-41bb-9acd-58f047fe9262 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:37.855275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742dfb-5727-41bb-9acd-58f047fe9262 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:37.855428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742dfb-5727-41bb-9acd-58f047fe9262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:37.922273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f5742dfb-5727-41bb-9acd-58f047fe9262 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:37.923571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f5742dfb-5727-41bb-9acd-58f047fe9262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:37.958259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:38.023748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 689/1387] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:38.071052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9a4aa-0648-4a2c-91a8-9220438a0795 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:38.072586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ece9a4aa-0648-4a2c-91a8-9220438a0795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:38.162856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:38.243729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9a4aa-0648-4a2c-91a8-9220438a0795 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:38.359225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c01bd-4703-4b79-b4ee-1bb744d9ec37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:38.359836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c01bd-4703-4b79-b4ee-1bb744d9ec37 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:38.360114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c01bd-4703-4b79-b4ee-1bb744d9ec37 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:38.360388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c01bd-4703-4b79-b4ee-1bb744d9ec37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:38.362018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3b0c01bd-4703-4b79-b4ee-1bb744d9ec37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:38.399883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:38.451432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c01bd-4703-4b79-b4ee-1bb744d9ec37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:38.452965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c01bd-4703-4b79-b4ee-1bb744d9ec37 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:38.471867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 699/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:40.496544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28324b0-99b8-46ac-9652-106540d14f8b None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:40.497065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28324b0-99b8-46ac-9652-106540d14f8b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:40.497153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28324b0-99b8-46ac-9652-106540d14f8b None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:40.498493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 690/1389] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:40.503629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828ee15-e38e-43b0-9dc6-1e5edaaa9262 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:40.504046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828ee15-e38e-43b0-9dc6-1e5edaaa9262 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:40.504237 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828ee15-e38e-43b0-9dc6-1e5edaaa9262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:40.570257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8828ee15-e38e-43b0-9dc6-1e5edaaa9262 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:40.571358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8828ee15-e38e-43b0-9dc6-1e5edaaa9262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:40.673919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 700/1390] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:40.733525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28324b0-99b8-46ac-9652-106540d14f8b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:40.737288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e28324b0-99b8-46ac-9652-106540d14f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:40.879366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28324b0-99b8-46ac-9652-106540d14f8b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:40.896060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:40.991685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13cbfa-7333-49b7-bcf2-3c55341c0678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:40.992283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13cbfa-7333-49b7-bcf2-3c55341c0678 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:40.992558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13cbfa-7333-49b7-bcf2-3c55341c0678 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:40.992823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13cbfa-7333-49b7-bcf2-3c55341c0678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:40.994455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0c13cbfa-7333-49b7-bcf2-3c55341c0678 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:41.082858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13cbfa-7333-49b7-bcf2-3c55341c0678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:41.084448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13cbfa-7333-49b7-bcf2-3c55341c0678 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:41.102735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 691/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:42.024338 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c17c-1d09-4c36-80ac-7ce41fc631a9 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.024908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c17c-1d09-4c36-80ac-7ce41fc631a9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.024908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c17c-1d09-4c36-80ac-7ce41fc631a9 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.026360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 701/1392] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:42.031664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d6973-cb31-4b06-9db8-b7d098befb57 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.032671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d6973-cb31-4b06-9db8-b7d098befb57 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.032774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d6973-cb31-4b06-9db8-b7d098befb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.058385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:42.100738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f18d6973-cb31-4b06-9db8-b7d098befb57 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:42.102215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f18d6973-cb31-4b06-9db8-b7d098befb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:42.199733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 692/1393] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:42.257856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f02ea-026e-4ad0-bd77-d513426945e7 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.258143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f02ea-026e-4ad0-bd77-d513426945e7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.258421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f02ea-026e-4ad0-bd77-d513426945e7 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.259758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 702/1394] 10.222.0.65 () {54 vars in 967 bytes} [Wed Aug 10 12:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:42.264823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d9ef-1af5-40aa-ae77-39f516808275 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.265117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d9ef-1af5-40aa-ae77-39f516808275 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.265426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d9ef-1af5-40aa-ae77-39f516808275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.345054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-68e1d9ef-1af5-40aa-ae77-39f516808275 None None] MFA Rules not processed for user `9be8c8c5f7f6417e9ab19ef5bce0f060`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:42.346221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-68e1d9ef-1af5-40aa-ae77-39f516808275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:42.386490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:42.422204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 693/1395] 10.222.0.65 () {58 vars in 1064 bytes} [Wed Aug 10 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:42.429741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f02ea-026e-4ad0-bd77-d513426945e7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:42.431260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-696f02ea-026e-4ad0-bd77-d513426945e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:42.445519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:42.536734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f02ea-026e-4ad0-bd77-d513426945e7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:42.615995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66385c11-4b3c-4b5a-a1b5-8d2c705932b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9be8c8c5f7f6417e9ab19ef5bce0f060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:42.616602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385c11-4b3c-4b5a-a1b5-8d2c705932b7 service glance] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:42.616883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385c11-4b3c-4b5a-a1b5-8d2c705932b7 service glance] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:42.617156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385c11-4b3c-4b5a-a1b5-8d2c705932b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:42.618787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-66385c11-4b3c-4b5a-a1b5-8d2c705932b7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:42.689120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:42.730734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66385c11-4b3c-4b5a-a1b5-8d2c705932b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:42.732312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66385c11-4b3c-4b5a-a1b5-8d2c705932b7 service glance] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:42.752608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 703/1396] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:43.125715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3760a9-92c9-44f2-816e-abfc4098f34e None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:43.126199 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3760a9-92c9-44f2-816e-abfc4098f34e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:43.126292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3760a9-92c9-44f2-816e-abfc4098f34e None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:43.127620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 694/1397] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:43.133188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82362b5-361c-4569-89cd-964f676cb8e5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:43.133656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82362b5-361c-4569-89cd-964f676cb8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:43.133760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82362b5-361c-4569-89cd-964f676cb8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:43.203366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-f82362b5-361c-4569-89cd-964f676cb8e5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:43.204567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f82362b5-361c-4569-89cd-964f676cb8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:43.304911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 704/1398] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:43.349230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3760a9-92c9-44f2-816e-abfc4098f34e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:43.350898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-de3760a9-92c9-44f2-816e-abfc4098f34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:43.507306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3760a9-92c9-44f2-816e-abfc4098f34e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:43.511097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab2740-c331-4d4f-840d-f2b5e93f42d3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:43.511392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab2740-c331-4d4f-840d-f2b5e93f42d3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:43.511673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab2740-c331-4d4f-840d-f2b5e93f42d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:43.539568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:43.555524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:43.581114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-77ab2740-c331-4d4f-840d-f2b5e93f42d3 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:43.582358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-77ab2740-c331-4d4f-840d-f2b5e93f42d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:43.626804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:43.627422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:43.627706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:43.627977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:43.629604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:43.674087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 705/1399] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:24:43.723148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6440fbf-9a43-41ec-9c43-4397691861c2 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:43.723581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6440fbf-9a43-41ec-9c43-4397691861c2 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:43.723860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6440fbf-9a43-41ec-9c43-4397691861c2 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:43.724287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:43.725255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 706/1400] 10.222.0.65 () {54 vars in 943 bytes} [Wed Aug 10 12:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:24:43.725947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:43.731690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa526f-c073-480e-992c-b200e3bd7b4d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:43.732025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa526f-c073-480e-992c-b200e3bd7b4d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:43.732360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa526f-c073-480e-992c-b200e3bd7b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:43.744557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 695/1401] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:43.813757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c7aa526f-c073-480e-992c-b200e3bd7b4d None None] MFA Rules not processed for user `1fb6a74548184806972a7f8deb30f789`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:43.815083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c7aa526f-c073-480e-992c-b200e3bd7b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:43.880798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:43.939573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 707/1402] 10.222.0.65 () {58 vars in 1040 bytes} [Wed Aug 10 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:24:43.949788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:43.951143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:44.006223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:44.040088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bb22c-5283-4efb-9487-f24ae579d6bb service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:44.143884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f811482-a1a4-4e3e-9963-c68fe87eb967 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:44.144564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811482-a1a4-4e3e-9963-c68fe87eb967 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:44.144755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811482-a1a4-4e3e-9963-c68fe87eb967 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:44.145027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f811482-a1a4-4e3e-9963-c68fe87eb967 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:44.146673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0f811482-a1a4-4e3e-9963-c68fe87eb967 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:44.227746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f811482-a1a4-4e3e-9963-c68fe87eb967 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:44.229437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f811482-a1a4-4e3e-9963-c68fe87eb967 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:44.233945 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:44.238591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 696/1403] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:24:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:24:45.792256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c935d-93fb-4f29-b3c2-ee318ea276cb None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.792786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c935d-93fb-4f29-b3c2-ee318ea276cb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.792919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c935d-93fb-4f29-b3c2-ee318ea276cb None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.795005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 708/1404] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:45.800960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce26138-2249-4df9-bdef-24ef90310fed None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:45.801445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce26138-2249-4df9-bdef-24ef90310fed None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:45.801552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce26138-2249-4df9-bdef-24ef90310fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:45.878106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9ce26138-2249-4df9-bdef-24ef90310fed None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:45.879264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9ce26138-2249-4df9-bdef-24ef90310fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:45.993944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 697/1405] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:46.042228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c935d-93fb-4f29-b3c2-ee318ea276cb None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:46.043927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e86c935d-93fb-4f29-b3c2-ee318ea276cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:46.205417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c935d-93fb-4f29-b3c2-ee318ea276cb None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:46.236887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:46.336556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546343b8-126c-4916-bff2-5b276a9e9fd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:46.338847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546343b8-126c-4916-bff2-5b276a9e9fd9 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:46.338847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546343b8-126c-4916-bff2-5b276a9e9fd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:46.338847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546343b8-126c-4916-bff2-5b276a9e9fd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:46.339373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-546343b8-126c-4916-bff2-5b276a9e9fd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:46.436572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546343b8-126c-4916-bff2-5b276a9e9fd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:46.438543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546343b8-126c-4916-bff2-5b276a9e9fd9 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:46.457344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 709/1406] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:48.496994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b180cd-1d46-4a93-9712-32099af19c59 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:48.497560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b180cd-1d46-4a93-9712-32099af19c59 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:48.497560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b180cd-1d46-4a93-9712-32099af19c59 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:48.502139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 698/1407] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:24:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:24:48.504401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac202e67-4946-475f-b5a3-9455c802f0c8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:48.504854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac202e67-4946-475f-b5a3-9455c802f0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:48.504948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac202e67-4946-475f-b5a3-9455c802f0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:48.549085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:48.579200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ac202e67-4946-475f-b5a3-9455c802f0c8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:24:48.580959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ac202e67-4946-475f-b5a3-9455c802f0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:48.695314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 710/1408] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:24:48.739957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b180cd-1d46-4a93-9712-32099af19c59 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:24:48.741586 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c7b180cd-1d46-4a93-9712-32099af19c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:48.832242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:48.905575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b180cd-1d46-4a93-9712-32099af19c59 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:24:49.029860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4283fc8-7eb9-423b-992c-66daae849408 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:24:49.030510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4283fc8-7eb9-423b-992c-66daae849408 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:24:49.030918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4283fc8-7eb9-423b-992c-66daae849408 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:24:49.031239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4283fc8-7eb9-423b-992c-66daae849408 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:24:49.032959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b4283fc8-7eb9-423b-992c-66daae849408 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:24:49.080673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:49.131841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4283fc8-7eb9-423b-992c-66daae849408 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:24:49.133562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4283fc8-7eb9-423b-992c-66daae849408 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:24:49.155778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 699/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:24:58.693415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=51 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:24:59.152431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=27 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:06.317760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbdcd6-51cb-4656-9a56-6cba835201a1 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:06.318308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbdcd6-51cb-4656-9a56-6cba835201a1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:06.318308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbdcd6-51cb-4656-9a56-6cba835201a1 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:06.319650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 711/1410] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 10 12:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:06.324746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc919f6-aa76-474e-8ded-f42e0a4c881f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:06.325165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc919f6-aa76-474e-8ded-f42e0a4c881f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:06.325297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc919f6-aa76-474e-8ded-f42e0a4c881f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:06.399112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ddc919f6-aa76-474e-8ded-f42e0a4c881f None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:06.400299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ddc919f6-aa76-474e-8ded-f42e0a4c881f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:06.503793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 700/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 10 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:06.518402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e1790-4367-488e-94de-955b20312056 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:06.518688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e1790-4367-488e-94de-955b20312056 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:06.518959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e1790-4367-488e-94de-955b20312056 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:06.520290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 712/1412] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 10 12:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:06.524882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166aea5-6281-4ede-9cf2-f7b9b4ded153 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:06.525163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166aea5-6281-4ede-9cf2-f7b9b4ded153 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:06.525467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166aea5-6281-4ede-9cf2-f7b9b4ded153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:06.600366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:06.603344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b166aea5-6281-4ede-9cf2-f7b9b4ded153 None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:06.605109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b166aea5-6281-4ede-9cf2-f7b9b4ded153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:06.717485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 701/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 10 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:09.159268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c3285-fd87-4597-955c-34ac5195232c None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:09.159798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c3285-fd87-4597-955c-34ac5195232c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:09.159798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c3285-fd87-4597-955c-34ac5195232c None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:09.161163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:09.166308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db5f45-d866-4900-883d-dcbf365b5058 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:09.166748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db5f45-d866-4900-883d-dcbf365b5058 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:09.166852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db5f45-d866-4900-883d-dcbf365b5058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:09.231560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f6db5f45-d866-4900-883d-dcbf365b5058 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:09.232757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f6db5f45-d866-4900-883d-dcbf365b5058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:09.286762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:09.329810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:09.371234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c3285-fd87-4597-955c-34ac5195232c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:09.372755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-368c3285-fd87-4597-955c-34ac5195232c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:09.518584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c3285-fd87-4597-955c-34ac5195232c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:09.593026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:09.634902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0088cb-8287-47c2-9613-640dd9aeb8f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:09.635514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0088cb-8287-47c2-9613-640dd9aeb8f3 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:09.635789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0088cb-8287-47c2-9613-640dd9aeb8f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:09.636058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0088cb-8287-47c2-9613-640dd9aeb8f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:09.637656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fd0088cb-8287-47c2-9613-640dd9aeb8f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:09.724122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0088cb-8287-47c2-9613-640dd9aeb8f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:09.725733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0088cb-8287-47c2-9613-640dd9aeb8f3 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:09.743774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 714/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:10.630948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb5220-6cc2-45a3-a1e4-7b14417da778 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:10.631406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb5220-6cc2-45a3-a1e4-7b14417da778 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:10.631509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb5220-6cc2-45a3-a1e4-7b14417da778 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:10.632921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:10.638058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1a261-ea1d-4c3f-8bb1-d12db603b927 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:10.638479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1a261-ea1d-4c3f-8bb1-d12db603b927 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:10.638629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1a261-ea1d-4c3f-8bb1-d12db603b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:10.714664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-f4d1a261-ea1d-4c3f-8bb1-d12db603b927 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:10.715856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f4d1a261-ea1d-4c3f-8bb1-d12db603b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:10.742766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:10.810843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:10.960932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb5220-6cc2-45a3-a1e4-7b14417da778 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:10.962481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-67bb5220-6cc2-45a3-a1e4-7b14417da778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:11.105131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb5220-6cc2-45a3-a1e4-7b14417da778 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:11.144524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:11.216542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb951bfe-2742-48b1-a23f-f7b15a145132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:11.217074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb951bfe-2742-48b1-a23f-f7b15a145132 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:11.217382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb951bfe-2742-48b1-a23f-f7b15a145132 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:11.217656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb951bfe-2742-48b1-a23f-f7b15a145132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:11.219237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cb951bfe-2742-48b1-a23f-f7b15a145132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:11.307447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb951bfe-2742-48b1-a23f-f7b15a145132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:11.309221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb951bfe-2742-48b1-a23f-f7b15a145132 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:11.328306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 704/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:12.197140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb4f8-c5a0-4e4f-96e3-c1cc627d3b83 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.197789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb4f8-c5a0-4e4f-96e3-c1cc627d3b83 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.197789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb4f8-c5a0-4e4f-96e3-c1cc627d3b83 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.199208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:12.204602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd2e5-3545-4de9-b2bb-527ef3524768 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.204989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd2e5-3545-4de9-b2bb-527ef3524768 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.205141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd2e5-3545-4de9-b2bb-527ef3524768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.258909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:12.281198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-49ebd2e5-3545-4de9-b2bb-527ef3524768 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:12.282471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-49ebd2e5-3545-4de9-b2bb-527ef3524768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:12.377776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:12.531028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eb4f8-c5a0-4e4f-96e3-c1cc627d3b83 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:12.532434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a14eb4f8-c5a0-4e4f-96e3-c1cc627d3b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:12.680761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eb4f8-c5a0-4e4f-96e3-c1cc627d3b83 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:12.693367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:12.800188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a989989-dbc0-4c10-be09-efc3029035c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:12.800700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a989989-dbc0-4c10-be09-efc3029035c0 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:12.800974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a989989-dbc0-4c10-be09-efc3029035c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:12.801243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a989989-dbc0-4c10-be09-efc3029035c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:12.802860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6a989989-dbc0-4c10-be09-efc3029035c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:12.894975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a989989-dbc0-4c10-be09-efc3029035c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:12.897128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a989989-dbc0-4c10-be09-efc3029035c0 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:12.917656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 717/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:13.804427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ebf89-0061-4466-9ea0-1f93018dc553 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:13.804996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ebf89-0061-4466-9ea0-1f93018dc553 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:13.804996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ebf89-0061-4466-9ea0-1f93018dc553 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:13.806361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:13.811470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c752b6-3b50-4167-9f11-b8a6fe868e04 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:13.811870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c752b6-3b50-4167-9f11-b8a6fe868e04 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:13.812044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c752b6-3b50-4167-9f11-b8a6fe868e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:13.833707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:13.890319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-78c752b6-3b50-4167-9f11-b8a6fe868e04 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:13.891460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-78c752b6-3b50-4167-9f11-b8a6fe868e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:13.986353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:14.134207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ebf89-0061-4466-9ea0-1f93018dc553 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:14.135732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-fe4ebf89-0061-4466-9ea0-1f93018dc553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:14.244458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:14.280101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ebf89-0061-4466-9ea0-1f93018dc553 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:14.393393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb38e65-63a3-4c43-be81-3e631f413174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:14.394006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38e65-63a3-4c43-be81-3e631f413174 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:14.394284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38e65-63a3-4c43-be81-3e631f413174 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:14.394556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38e65-63a3-4c43-be81-3e631f413174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:14.396151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-afb38e65-63a3-4c43-be81-3e631f413174 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:14.472111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:14.486174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb38e65-63a3-4c43-be81-3e631f413174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:14.487764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb38e65-63a3-4c43-be81-3e631f413174 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:14.505923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 707/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:15.366760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680d78-8d13-46d2-a839-a5751b7c91d8 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:15.367304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680d78-8d13-46d2-a839-a5751b7c91d8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:15.367304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680d78-8d13-46d2-a839-a5751b7c91d8 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:15.368635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:25:15.373493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ab485-110d-4d72-a013-33e7dda07ed0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:15.373881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ab485-110d-4d72-a013-33e7dda07ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:15.374046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ab485-110d-4d72-a013-33e7dda07ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:15.448781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-043ab485-110d-4d72-a013-33e7dda07ed0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:15.450041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-043ab485-110d-4d72-a013-33e7dda07ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:15.544883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:15.694300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680d78-8d13-46d2-a839-a5751b7c91d8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:15.696331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5a680d78-8d13-46d2-a839-a5751b7c91d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:15.779776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:15.841147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680d78-8d13-46d2-a839-a5751b7c91d8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:15.951185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51736a3a-4dc8-4a15-8abd-0505655802c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:15.951723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51736a3a-4dc8-4a15-8abd-0505655802c4 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:15.952001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51736a3a-4dc8-4a15-8abd-0505655802c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:15.952269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51736a3a-4dc8-4a15-8abd-0505655802c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:15.953886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-51736a3a-4dc8-4a15-8abd-0505655802c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:16.000561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:16.042213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51736a3a-4dc8-4a15-8abd-0505655802c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:16.043828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51736a3a-4dc8-4a15-8abd-0505655802c4 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:16.061463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 720/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:25:16.858607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7b755-8692-4884-9200-572231d8808a None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:16.859140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7b755-8692-4884-9200-572231d8808a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:16.859140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7b755-8692-4884-9200-572231d8808a None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:16.860423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:16.865911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f2ea-fdec-4ec4-a654-33f75d8cc39e None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:16.866333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f2ea-fdec-4ec4-a654-33f75d8cc39e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:16.866446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f2ea-fdec-4ec4-a654-33f75d8cc39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:16.942456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-44a7f2ea-fdec-4ec4-a654-33f75d8cc39e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:16.943662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-44a7f2ea-fdec-4ec4-a654-33f75d8cc39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:17.038374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:17.187791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7b755-8692-4884-9200-572231d8808a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:17.189359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-46b7b755-8692-4884-9200-572231d8808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:17.238150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:17.333792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7b755-8692-4884-9200-572231d8808a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:17.444608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6aefea-72e2-4716-8c3b-2398f656551e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:17.445135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6aefea-72e2-4716-8c3b-2398f656551e service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:17.445445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6aefea-72e2-4716-8c3b-2398f656551e service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:17.445722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6aefea-72e2-4716-8c3b-2398f656551e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:17.447304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ce6aefea-72e2-4716-8c3b-2398f656551e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:17.454093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:17.536876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6aefea-72e2-4716-8c3b-2398f656551e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:17.538537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6aefea-72e2-4716-8c3b-2398f656551e service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:17.555917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 710/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:18.355253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d052-6b1d-4043-90df-400584a799a1 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.355755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d052-6b1d-4043-90df-400584a799a1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.355869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60d052-6b1d-4043-90df-400584a799a1 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.357162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:18.362579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ddc87-3655-4a9d-9bbd-4265f5068a27 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.363026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ddc87-3655-4a9d-9bbd-4265f5068a27 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.363155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ddc87-3655-4a9d-9bbd-4265f5068a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.437210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-935ddc87-3655-4a9d-9bbd-4265f5068a27 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:18.438536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-935ddc87-3655-4a9d-9bbd-4265f5068a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:18.505064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:18.533936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:18.686378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60d052-6b1d-4043-90df-400584a799a1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:18.687945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0a60d052-6b1d-4043-90df-400584a799a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:18.705465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:18.830278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60d052-6b1d-4043-90df-400584a799a1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:18.916151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:18.941828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb0cb1-3dd2-4539-bf4f-beb47365e3d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:18.942429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb0cb1-3dd2-4539-bf4f-beb47365e3d6 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:18.942703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb0cb1-3dd2-4539-bf4f-beb47365e3d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:18.942971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb0cb1-3dd2-4539-bf4f-beb47365e3d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:18.944546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-aedb0cb1-3dd2-4539-bf4f-beb47365e3d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:19.033106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb0cb1-3dd2-4539-bf4f-beb47365e3d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:19.034738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb0cb1-3dd2-4539-bf4f-beb47365e3d6 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:19.052376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 723/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:19.853963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a4dce-2b57-4b60-a03d-88092db54837 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:19.854468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a4dce-2b57-4b60-a03d-88092db54837 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:19.854468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a4dce-2b57-4b60-a03d-88092db54837 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:19.855783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:19.860722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa79162-4b3d-4c36-91c8-7395d8b27b2f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:19.861163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa79162-4b3d-4c36-91c8-7395d8b27b2f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:19.861301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa79162-4b3d-4c36-91c8-7395d8b27b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:19.936488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8fa79162-4b3d-4c36-91c8-7395d8b27b2f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:19.937720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8fa79162-4b3d-4c36-91c8-7395d8b27b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:19.978482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:20.032347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:20.183305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a4dce-2b57-4b60-a03d-88092db54837 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:20.184809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b45a4dce-2b57-4b60-a03d-88092db54837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:20.331327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a4dce-2b57-4b60-a03d-88092db54837 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:20.384923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:20.445231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68e61d-7cd2-4750-bdb0-f38d2a590d0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:20.445782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68e61d-7cd2-4750-bdb0-f38d2a590d0b service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:20.446055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68e61d-7cd2-4750-bdb0-f38d2a590d0b service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:20.446324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68e61d-7cd2-4750-bdb0-f38d2a590d0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:20.447901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5c68e61d-7cd2-4750-bdb0-f38d2a590d0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:20.536101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68e61d-7cd2-4750-bdb0-f38d2a590d0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:20.537759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68e61d-7cd2-4750-bdb0-f38d2a590d0b service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:20.556390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 713/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:21.352273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ef6c-7e41-4d81-ac2e-0afea0d6853a None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:21.352822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ef6c-7e41-4d81-ac2e-0afea0d6853a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:21.352822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ef6c-7e41-4d81-ac2e-0afea0d6853a None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:21.354193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:21.359043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44cc55-e2b6-4514-a388-6f985f968a32 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:21.359464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44cc55-e2b6-4514-a388-6f985f968a32 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:21.359637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44cc55-e2b6-4514-a388-6f985f968a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:21.438888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-da44cc55-e2b6-4514-a388-6f985f968a32 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:21.440059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-da44cc55-e2b6-4514-a388-6f985f968a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:21.442106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:21.535901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:21.683911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745ef6c-7e41-4d81-ac2e-0afea0d6853a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:21.685475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8745ef6c-7e41-4d81-ac2e-0afea0d6853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:21.828692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745ef6c-7e41-4d81-ac2e-0afea0d6853a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:21.856101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:21.941580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f019-7576-447c-82ee-fa3f58d4098f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:21.942104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f019-7576-447c-82ee-fa3f58d4098f service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:21.942381 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f019-7576-447c-82ee-fa3f58d4098f service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:21.942652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f019-7576-447c-82ee-fa3f58d4098f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:21.944240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-76e8f019-7576-447c-82ee-fa3f58d4098f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:22.031867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8f019-7576-447c-82ee-fa3f58d4098f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:22.033555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8f019-7576-447c-82ee-fa3f58d4098f service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:22.051079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 726/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:23.943695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4830-5467-405b-af83-11738125f63d None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:23.944241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4830-5467-405b-af83-11738125f63d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:23.944241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4830-5467-405b-af83-11738125f63d None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:23.945648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:23.950756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f7da-d68c-4f57-939e-98c5cedf3bcc None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:23.951183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f7da-d68c-4f57-939e-98c5cedf3bcc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:23.951305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f7da-d68c-4f57-939e-98c5cedf3bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:23.993117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:24.018398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8f45f7da-d68c-4f57-939e-98c5cedf3bcc None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:24.019592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8f45f7da-d68c-4f57-939e-98c5cedf3bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:24.112857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:24.150961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4830-5467-405b-af83-11738125f63d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:24.152464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a3dd4830-5467-405b-af83-11738125f63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:24.254398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4830-5467-405b-af83-11738125f63d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:24.286021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:24.330742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5562663-6535-41e0-bd0d-4244e689dceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:24.331377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562663-6535-41e0-bd0d-4244e689dceb service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:24.331658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562663-6535-41e0-bd0d-4244e689dceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:24.331932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562663-6535-41e0-bd0d-4244e689dceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:24.333557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a5562663-6535-41e0-bd0d-4244e689dceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:24.442178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5562663-6535-41e0-bd0d-4244e689dceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:24.443626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5562663-6535-41e0-bd0d-4244e689dceb service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:24.461319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 716/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:26.592113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b25978-93c4-47bf-bba3-8983d5fd85e0 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:26.592589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b25978-93c4-47bf-bba3-8983d5fd85e0 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:26.592706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b25978-93c4-47bf-bba3-8983d5fd85e0 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:26.594074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:26.599996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a772a-f06d-4685-bf14-77cb12823fd0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:26.600470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a772a-f06d-4685-bf14-77cb12823fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:26.600755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a772a-f06d-4685-bf14-77cb12823fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:26.666501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4b9a772a-f06d-4685-bf14-77cb12823fd0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:26.667702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4b9a772a-f06d-4685-bf14-77cb12823fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:26.680786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:26.763921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:26.802205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b25978-93c4-47bf-bba3-8983d5fd85e0 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:26.803750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c8b25978-93c4-47bf-bba3-8983d5fd85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:26.906030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b25978-93c4-47bf-bba3-8983d5fd85e0 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:26.918371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:26.979471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27025405-a97c-4cad-aaaa-dc33c9367ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:26.980077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27025405-a97c-4cad-aaaa-dc33c9367ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:26.980359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27025405-a97c-4cad-aaaa-dc33c9367ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:26.980657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27025405-a97c-4cad-aaaa-dc33c9367ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:26.982415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-27025405-a97c-4cad-aaaa-dc33c9367ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:27.092658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27025405-a97c-4cad-aaaa-dc33c9367ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:27.094137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27025405-a97c-4cad-aaaa-dc33c9367ab2 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:27.112834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 729/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:29.389382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b31f9-b882-4dae-93c8-3f9463caf4ba None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:29.389963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b31f9-b882-4dae-93c8-3f9463caf4ba None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:29.389963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b31f9-b882-4dae-93c8-3f9463caf4ba None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:29.391324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:29.396664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214dd1e-b69e-45e1-a525-35dc07f74a70 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:29.397081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214dd1e-b69e-45e1-a525-35dc07f74a70 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:29.397217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214dd1e-b69e-45e1-a525-35dc07f74a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:29.445687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:29.463155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b214dd1e-b69e-45e1-a525-35dc07f74a70 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:29.464349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b214dd1e-b69e-45e1-a525-35dc07f74a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:29.568922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:29.607588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b31f9-b882-4dae-93c8-3f9463caf4ba None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:29.609134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-710b31f9-b882-4dae-93c8-3f9463caf4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:29.712266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b31f9-b882-4dae-93c8-3f9463caf4ba None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:29.752602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:29.786851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6ee2d-ec1b-4e7d-931e-a21045c083cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:29.787457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ee2d-ec1b-4e7d-931e-a21045c083cf service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:29.787739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ee2d-ec1b-4e7d-931e-a21045c083cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:29.788012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ee2d-ec1b-4e7d-931e-a21045c083cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:29.789644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-63c6ee2d-ec1b-4e7d-931e-a21045c083cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:29.897734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6ee2d-ec1b-4e7d-931e-a21045c083cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:29.899167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6ee2d-ec1b-4e7d-931e-a21045c083cf service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:29.918899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 719/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:32.022613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac09399-6ff0-4d4a-9445-577181b79f78 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:32.023152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac09399-6ff0-4d4a-9445-577181b79f78 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:32.023152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac09399-6ff0-4d4a-9445-577181b79f78 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:32.024506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:32.029833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d949-d712-4bde-a000-cd7a253ca287 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:32.030236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d949-d712-4bde-a000-cd7a253ca287 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:32.030375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2d949-d712-4bde-a000-cd7a253ca287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:32.094960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4ee2d949-d712-4bde-a000-cd7a253ca287 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:32.096148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4ee2d949-d712-4bde-a000-cd7a253ca287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:32.117006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:32.194441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:32.233147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac09399-6ff0-4d4a-9445-577181b79f78 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:32.234717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3ac09399-6ff0-4d4a-9445-577181b79f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:32.337795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac09399-6ff0-4d4a-9445-577181b79f78 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:32.357105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:32.409426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f1824-9fe6-4690-abec-adb9355d0155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:32.410024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f1824-9fe6-4690-abec-adb9355d0155 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:32.410299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f1824-9fe6-4690-abec-adb9355d0155 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:32.410568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f1824-9fe6-4690-abec-adb9355d0155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:32.412136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7a0f1824-9fe6-4690-abec-adb9355d0155 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:32.519655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f1824-9fe6-4690-abec-adb9355d0155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:32.521106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f1824-9fe6-4690-abec-adb9355d0155 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:32.539460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 732/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:34.670714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2957f-ced8-4379-b457-db3e01824147 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:34.671254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2957f-ced8-4379-b457-db3e01824147 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:34.671254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2957f-ced8-4379-b457-db3e01824147 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:34.672632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:34.678231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66ff6-fc64-4bf9-81c3-e4988bae0418 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:34.678648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66ff6-fc64-4bf9-81c3-e4988bae0418 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:34.678785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66ff6-fc64-4bf9-81c3-e4988bae0418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:34.742516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:34.745120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7bb66ff6-fc64-4bf9-81c3-e4988bae0418 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:34.746977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7bb66ff6-fc64-4bf9-81c3-e4988bae0418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:34.840639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:34.878809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2957f-ced8-4379-b457-db3e01824147 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:34.880369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f3a2957f-ced8-4379-b457-db3e01824147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:34.982827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2957f-ced8-4379-b457-db3e01824147 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:35.030253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:35.055256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bbc3c-3760-4943-8778-634bc47dc26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:35.055850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bbc3c-3760-4943-8778-634bc47dc26f service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:35.056128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bbc3c-3760-4943-8778-634bc47dc26f service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:35.056397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bbc3c-3760-4943-8778-634bc47dc26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:35.057998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2b0bbc3c-3760-4943-8778-634bc47dc26f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:35.164260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bbc3c-3760-4943-8778-634bc47dc26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:35.165721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bbc3c-3760-4943-8778-634bc47dc26f service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:35.183441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 722/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:37.235486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2fed2-e18e-476a-a5d2-af2e0da4a2a3 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:37.236049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2fed2-e18e-476a-a5d2-af2e0da4a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:37.236049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2fed2-e18e-476a-a5d2-af2e0da4a2a3 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:37.237466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:37.242630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00850d-08e5-420e-861c-7287f074dccd None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:37.243070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00850d-08e5-420e-861c-7287f074dccd None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:37.243176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00850d-08e5-420e-861c-7287f074dccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:37.309902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4d00850d-08e5-420e-861c-7287f074dccd None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:37.311186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4d00850d-08e5-420e-861c-7287f074dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:37.341384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:37.408499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:37.447289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2fed2-e18e-476a-a5d2-af2e0da4a2a3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:37.448820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2fa2fed2-e18e-476a-a5d2-af2e0da4a2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:37.551284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2fed2-e18e-476a-a5d2-af2e0da4a2a3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:37.579375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:37.624772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201978-1499-4e0f-97a2-47eebe7fae2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:37.625414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201978-1499-4e0f-97a2-47eebe7fae2b service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:37.625700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201978-1499-4e0f-97a2-47eebe7fae2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:37.625967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201978-1499-4e0f-97a2-47eebe7fae2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:37.628178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-be201978-1499-4e0f-97a2-47eebe7fae2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:37.734489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201978-1499-4e0f-97a2-47eebe7fae2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:37.736597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201978-1499-4e0f-97a2-47eebe7fae2b service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:37.754151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 735/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:39.807407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835184f-dc4b-45b3-9b93-319d71177f8c None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:39.807931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835184f-dc4b-45b3-9b93-319d71177f8c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:39.807931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835184f-dc4b-45b3-9b93-319d71177f8c None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:39.809218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:25:39.814674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0e1b-ff4e-45c1-8563-d0e2bf671f9c None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:39.815058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0e1b-ff4e-45c1-8563-d0e2bf671f9c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:39.815226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0e1b-ff4e-45c1-8563-d0e2bf671f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:39.880700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a54a0e1b-ff4e-45c1-8563-d0e2bf671f9c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:39.881942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a54a0e1b-ff4e-45c1-8563-d0e2bf671f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:39.890330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:39.978571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:40.016445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835184f-dc4b-45b3-9b93-319d71177f8c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:40.018593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3835184f-dc4b-45b3-9b93-319d71177f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:40.118062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835184f-dc4b-45b3-9b93-319d71177f8c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:40.171545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:40.190006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b93763-ae7c-425d-8e9a-4be3c53718c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:40.190603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b93763-ae7c-425d-8e9a-4be3c53718c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:40.190880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b93763-ae7c-425d-8e9a-4be3c53718c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:40.191144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b93763-ae7c-425d-8e9a-4be3c53718c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:40.192706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-98b93763-ae7c-425d-8e9a-4be3c53718c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:40.300338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b93763-ae7c-425d-8e9a-4be3c53718c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:40.301818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b93763-ae7c-425d-8e9a-4be3c53718c1 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:40.321927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 725/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:42.379289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c9dce-cafa-4baa-bdf1-5f6de6196b8e None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:42.379809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c9dce-cafa-4baa-bdf1-5f6de6196b8e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:42.379907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c9dce-cafa-4baa-bdf1-5f6de6196b8e None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:42.381385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:42.386339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dadb6-450f-4f7c-bd33-c73a1b11b6e0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:42.386775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dadb6-450f-4f7c-bd33-c73a1b11b6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:42.386869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dadb6-450f-4f7c-bd33-c73a1b11b6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:42.452789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-432dadb6-450f-4f7c-bd33-c73a1b11b6e0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:42.454006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-432dadb6-450f-4f7c-bd33-c73a1b11b6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:42.491479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:42.549631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:42.587571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c9dce-cafa-4baa-bdf1-5f6de6196b8e None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:42.589080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7a9c9dce-cafa-4baa-bdf1-5f6de6196b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:42.692308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c9dce-cafa-4baa-bdf1-5f6de6196b8e None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:42.730011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:42.765957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4acd46-5e24-4f90-8809-22af9b0e9ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:42.766732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4acd46-5e24-4f90-8809-22af9b0e9ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:42.767021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4acd46-5e24-4f90-8809-22af9b0e9ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:42.767295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4acd46-5e24-4f90-8809-22af9b0e9ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:42.768901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-cf4acd46-5e24-4f90-8809-22af9b0e9ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:42.876630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4acd46-5e24-4f90-8809-22af9b0e9ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:42.878056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4acd46-5e24-4f90-8809-22af9b0e9ed3 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:42.895598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 738/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:44.916200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224c45-c815-49bb-9906-ddc46447f745 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:44.916745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224c45-c815-49bb-9906-ddc46447f745 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:44.916745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224c45-c815-49bb-9906-ddc46447f745 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:44.918137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:44.923968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f76a03-1f78-43b9-b3ed-d2ada6dbfe79 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:44.924374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f76a03-1f78-43b9-b3ed-d2ada6dbfe79 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:44.924495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f76a03-1f78-43b9-b3ed-d2ada6dbfe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:44.990490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-51f76a03-1f78-43b9-b3ed-d2ada6dbfe79 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:44.991688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-51f76a03-1f78-43b9-b3ed-d2ada6dbfe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:45.009906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:45.087871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:45.124727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224c45-c815-49bb-9906-ddc46447f745 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:45.126981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4b224c45-c815-49bb-9906-ddc46447f745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:45.228699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224c45-c815-49bb-9906-ddc46447f745 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:45.295135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:45.302993 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f66642-3f99-4adc-a36f-f2f3d1c55f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:45.303597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f66642-3f99-4adc-a36f-f2f3d1c55f51 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:45.303871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f66642-3f99-4adc-a36f-f2f3d1c55f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:45.304138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f66642-3f99-4adc-a36f-f2f3d1c55f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:45.305731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-02f66642-3f99-4adc-a36f-f2f3d1c55f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:45.413343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f66642-3f99-4adc-a36f-f2f3d1c55f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:45.414768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f66642-3f99-4adc-a36f-f2f3d1c55f51 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:45.433261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 728/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:47.433242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1c29-a6f1-4212-83e3-5b9c4df1ca21 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:47.433829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1c29-a6f1-4212-83e3-5b9c4df1ca21 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:47.433829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1c29-a6f1-4212-83e3-5b9c4df1ca21 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:47.435169 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:25:47.440622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0894-4da5-407d-baae-b411abf1351e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:47.441031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0894-4da5-407d-baae-b411abf1351e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:47.441178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa0894-4da5-407d-baae-b411abf1351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:47.506450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-23aa0894-4da5-407d-baae-b411abf1351e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:25:47.507631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-23aa0894-4da5-407d-baae-b411abf1351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:47.555313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:47.605405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:25:47.642414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1c29-a6f1-4212-83e3-5b9c4df1ca21 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:25:47.643928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fc2d1c29-a6f1-4212-83e3-5b9c4df1ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:47.745569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1c29-a6f1-4212-83e3-5b9c4df1ca21 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:25:47.793953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:47.820701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8010616-0fe1-4779-8b61-4a922f5b2328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:25:47.821332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8010616-0fe1-4779-8b61-4a922f5b2328 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:25:47.821613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8010616-0fe1-4779-8b61-4a922f5b2328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:25:47.821884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8010616-0fe1-4779-8b61-4a922f5b2328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:25:47.823458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b8010616-0fe1-4779-8b61-4a922f5b2328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:25:48.051737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8010616-0fe1-4779-8b61-4a922f5b2328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:25:48.053069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8010616-0fe1-4779-8b61-4a922f5b2328 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:25:48.073188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 741/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:25:57.603579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=22 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:25:58.070824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=66 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:07.412537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce7a05-bfe5-40fb-a892-e76026314256 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:07.413094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce7a05-bfe5-40fb-a892-e76026314256 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:07.413200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce7a05-bfe5-40fb-a892-e76026314256 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:07.414614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 730/1471] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:07.419834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598520b-0864-40c7-b12e-d1ab3f0707b5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:07.420225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598520b-0864-40c7-b12e-d1ab3f0707b5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:07.420401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598520b-0864-40c7-b12e-d1ab3f0707b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:07.484268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9598520b-0864-40c7-b12e-d1ab3f0707b5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:07.486040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9598520b-0864-40c7-b12e-d1ab3f0707b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:07.579921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 742/1472] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:07.663926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3569203-0c96-4063-8c45-707ac2ea61ea None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:07.664153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3569203-0c96-4063-8c45-707ac2ea61ea None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:07.664428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3569203-0c96-4063-8c45-707ac2ea61ea None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:07.665798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 731/1473] 10.222.0.65 () {54 vars in 967 bytes} [Wed Aug 10 12:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:07.670474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8244f3-7039-4996-82bd-c0a19eeb3602 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:07.670763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8244f3-7039-4996-82bd-c0a19eeb3602 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:07.671037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8244f3-7039-4996-82bd-c0a19eeb3602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:07.747335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-0b8244f3-7039-4996-82bd-c0a19eeb3602 None None] MFA Rules not processed for user `9be8c8c5f7f6417e9ab19ef5bce0f060`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:07.748531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0b8244f3-7039-4996-82bd-c0a19eeb3602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:07.768809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:07.824928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 743/1474] 10.222.0.65 () {58 vars in 1064 bytes} [Wed Aug 10 12:26:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:07.832496 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3569203-0c96-4063-8c45-707ac2ea61ea None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:07.834026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c3569203-0c96-4063-8c45-707ac2ea61ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:07.936428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3569203-0c96-4063-8c45-707ac2ea61ea None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:08.007735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8cd15-a7ec-41f6-874d-23620cd0c695 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9be8c8c5f7f6417e9ab19ef5bce0f060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:08.008773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8cd15-a7ec-41f6-874d-23620cd0c695 service glance] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:08.009320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8cd15-a7ec-41f6-874d-23620cd0c695 service glance] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:08.009877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8cd15-a7ec-41f6-874d-23620cd0c695 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:08.013002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e2f8cd15-a7ec-41f6-874d-23620cd0c695 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:08.072670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:08.127829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8cd15-a7ec-41f6-874d-23620cd0c695 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:08.129290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8cd15-a7ec-41f6-874d-23620cd0c695 service glance] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:08.148073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 732/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:26:10.218227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27981e57-2b46-4ad5-b370-18c62dac67cb None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:10.219308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27981e57-2b46-4ad5-b370-18c62dac67cb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:10.219308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27981e57-2b46-4ad5-b370-18c62dac67cb None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:10.220632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 744/1476] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:10.226207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a7e0-1b05-404d-99ba-94f947a1b63d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:10.226660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a7e0-1b05-404d-99ba-94f947a1b63d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:10.226764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a7e0-1b05-404d-99ba-94f947a1b63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:10.292614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b822a7e0-1b05-404d-99ba-94f947a1b63d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:10.293843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b822a7e0-1b05-404d-99ba-94f947a1b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:10.391906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 733/1477] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:10.415345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27981e57-2b46-4ad5-b370-18c62dac67cb None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:10.416876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-27981e57-2b46-4ad5-b370-18c62dac67cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:10.539504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27981e57-2b46-4ad5-b370-18c62dac67cb None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:10.586620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:10.630798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e0dc7-f794-4ba1-a3c8-c686fe6a8432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:10.631433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e0dc7-f794-4ba1-a3c8-c686fe6a8432 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:10.631702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e0dc7-f794-4ba1-a3c8-c686fe6a8432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:10.631961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e0dc7-f794-4ba1-a3c8-c686fe6a8432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:10.632588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e0dc7-f794-4ba1-a3c8-c686fe6a8432 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:10.634342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-de5e0dc7-f794-4ba1-a3c8-c686fe6a8432 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:26:10.634342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:26:10.634342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:26:10.635473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e0dc7-f794-4ba1-a3c8-c686fe6a8432 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:10.653943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 745/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 10 12:26:10 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:26:12.673673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0150b-dabf-405e-921c-5c2147e0e5b2 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:12.674216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0150b-dabf-405e-921c-5c2147e0e5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:12.674216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0150b-dabf-405e-921c-5c2147e0e5b2 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:12.675601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 734/1479] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:12.680843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd6c21-4755-4ea8-ba5f-660acd69f336 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:12.681287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd6c21-4755-4ea8-ba5f-660acd69f336 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:12.681405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd6c21-4755-4ea8-ba5f-660acd69f336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:12.745459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e9dd6c21-4755-4ea8-ba5f-660acd69f336 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:12.746679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e9dd6c21-4755-4ea8-ba5f-660acd69f336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:12.842677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 746/1480] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:12.886332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0150b-dabf-405e-921c-5c2147e0e5b2 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:12.887980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0fa0150b-dabf-405e-921c-5c2147e0e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:12.897708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:13.032550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0150b-dabf-405e-921c-5c2147e0e5b2 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:13.112778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:13.145246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d75b2b-b15c-4ac4-b595-1b3f15704cdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:13.145814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75b2b-b15c-4ac4-b595-1b3f15704cdf service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:13.146088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75b2b-b15c-4ac4-b595-1b3f15704cdf service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:13.146355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75b2b-b15c-4ac4-b595-1b3f15704cdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:13.147920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b7d75b2b-b15c-4ac4-b595-1b3f15704cdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:13.235158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d75b2b-b15c-4ac4-b595-1b3f15704cdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:13.237322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d75b2b-b15c-4ac4-b595-1b3f15704cdf service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:13.254877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 735/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:26:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:26:15.421542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0969-df30-4bf7-aed5-073829d51daa None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:15.422129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0969-df30-4bf7-aed5-073829d51daa None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:15.422129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0969-df30-4bf7-aed5-073829d51daa None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:15.423467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 747/1482] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:15.428563 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323541-3dac-45af-85de-3a948cd0e3ef None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:15.428959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323541-3dac-45af-85de-3a948cd0e3ef None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:15.429112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323541-3dac-45af-85de-3a948cd0e3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:15.493692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6c323541-3dac-45af-85de-3a948cd0e3ef None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:15.494893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6c323541-3dac-45af-85de-3a948cd0e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:15.540780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:15.591107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 736/1483] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:15.628549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0969-df30-4bf7-aed5-073829d51daa None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:15.630145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7bde0969-df30-4bf7-aed5-073829d51daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:15.653338 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:15.732036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0969-df30-4bf7-aed5-073829d51daa None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:15.808085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a26c0-032a-4586-990a-7a992a5b4939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:15.808684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a26c0-032a-4586-990a-7a992a5b4939 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:15.808966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a26c0-032a-4586-990a-7a992a5b4939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:15.809236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a26c0-032a-4586-990a-7a992a5b4939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:15.810850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7f3a26c0-032a-4586-990a-7a992a5b4939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:15.884000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:15.917868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a26c0-032a-4586-990a-7a992a5b4939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:15.919296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a26c0-032a-4586-990a-7a992a5b4939 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:15.937432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 748/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:26:17.976455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c8a5-fc02-4096-9fb0-ac203778d8a5 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:17.977006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c8a5-fc02-4096-9fb0-ac203778d8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:17.977006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c8a5-fc02-4096-9fb0-ac203778d8a5 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:17.978392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 737/1485] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:17.983736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c6857-b518-44b5-a8ed-31235c8c1ab4 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:17.984162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c6857-b518-44b5-a8ed-31235c8c1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:17.984278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c6857-b518-44b5-a8ed-31235c8c1ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:18.049242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a59c6857-b518-44b5-a8ed-31235c8c1ab4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:18.050467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a59c6857-b518-44b5-a8ed-31235c8c1ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:18.146482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 749/1486] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:18.184526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c8a5-fc02-4096-9fb0-ac203778d8a5 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:18.186155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2012c8a5-fc02-4096-9fb0-ac203778d8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:18.290734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c8a5-fc02-4096-9fb0-ac203778d8a5 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:18.362935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:18.364885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b1aa0-1a0a-4434-b1d9-6eea714a238d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:18.365714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1aa0-1a0a-4434-b1d9-6eea714a238d service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:18.366183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1aa0-1a0a-4434-b1d9-6eea714a238d service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:18.366701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1aa0-1a0a-4434-b1d9-6eea714a238d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:18.368287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a67b1aa0-1a0a-4434-b1d9-6eea714a238d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:18.474797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b1aa0-1a0a-4434-b1d9-6eea714a238d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:18.476504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b1aa0-1a0a-4434-b1d9-6eea714a238d service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:18.495122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 738/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:26:20.583242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7d95d-ed81-4e82-aa0d-231ae5fea93b None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:20.583736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7d95d-ed81-4e82-aa0d-231ae5fea93b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:20.583736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7d95d-ed81-4e82-aa0d-231ae5fea93b None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:20.585075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 750/1488] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:20.590199 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5712f-20cf-4ccf-805c-29f3d06d70a5 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:20.590619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5712f-20cf-4ccf-805c-29f3d06d70a5 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:20.590734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5712f-20cf-4ccf-805c-29f3d06d70a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:20.656671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-43d5712f-20cf-4ccf-805c-29f3d06d70a5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:20.658391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-43d5712f-20cf-4ccf-805c-29f3d06d70a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:20.710525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:20.755428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 739/1489] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:20.795381 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7d95d-ed81-4e82-aa0d-231ae5fea93b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:20.796902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d3b7d95d-ed81-4e82-aa0d-231ae5fea93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:20.832737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:20.900111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7d95d-ed81-4e82-aa0d-231ae5fea93b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:20.971559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca6f11-3e86-419f-851d-1e885a032e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:20.972158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca6f11-3e86-419f-851d-1e885a032e1e service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:20.972438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca6f11-3e86-419f-851d-1e885a032e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:20.972708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca6f11-3e86-419f-851d-1e885a032e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:20.974319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d3ca6f11-3e86-419f-851d-1e885a032e1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:21.053230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:21.081966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca6f11-3e86-419f-851d-1e885a032e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:21.083399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca6f11-3e86-419f-851d-1e885a032e1e service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:21.101526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 751/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:26:23.290936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e6cae-9edc-499e-8700-b19660dccd5f None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:23.291467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e6cae-9edc-499e-8700-b19660dccd5f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:23.291467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e6cae-9edc-499e-8700-b19660dccd5f None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:23.292775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 740/1491] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 10 12:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:26:23.297953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbb5c3-79f7-41ef-92d0-846be4c8bbdb None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:23.298351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbb5c3-79f7-41ef-92d0-846be4c8bbdb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:23.298496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbb5c3-79f7-41ef-92d0-846be4c8bbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:23.377645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b6dbb5c3-79f7-41ef-92d0-846be4c8bbdb None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:26:23.378852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b6dbb5c3-79f7-41ef-92d0-846be4c8bbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:23.478225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 752/1492] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 10 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:26:23.519665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e6cae-9edc-499e-8700-b19660dccd5f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:26:23.521209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-164e6cae-9edc-499e-8700-b19660dccd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:23.630102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e6cae-9edc-499e-8700-b19660dccd5f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:26:23.703609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f6173-168c-4a27-b307-3e200622f38a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:26:23.704815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f6173-168c-4a27-b307-3e200622f38a service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:26:23.705098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f6173-168c-4a27-b307-3e200622f38a service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:26:23.705503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f6173-168c-4a27-b307-3e200622f38a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:26:23.707134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-aa3f6173-168c-4a27-b307-3e200622f38a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:26:23.716224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:23.818796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f6173-168c-4a27-b307-3e200622f38a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:26:23.820548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f6173-168c-4a27-b307-3e200622f38a service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:26:23.839340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 741/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:26:33.476392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=82 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:26:33.837094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=50 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:34.264795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:34.265523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:34.265523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:34.340452 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:34.341669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:34.437711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 753/1494] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:27:34.445849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cd338-c177-4890-a2fc-c9eb4163e3dd None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:34.446311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cd338-c177-4890-a2fc-c9eb4163e3dd None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:34.446447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cd338-c177-4890-a2fc-c9eb4163e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:34.520979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a69cd338-c177-4890-a2fc-c9eb4163e3dd None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:34.522140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a69cd338-c177-4890-a2fc-c9eb4163e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:34.620410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 742/1495] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:27:34.625720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:34.626951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:34.638266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87c5ce-1088-4293-8815-3315e3feb2ed None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:34.698753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:34.734081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:34.734674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:34.734949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:34.735217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:34.735858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:27:34.736154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:34.737152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:34.737152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:27:34.737152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:27:34.738063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:34.748862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 754/1496] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:27:34.757055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cd338-c177-4890-a2fc-c9eb4163e3dd None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:34.766320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cd338-c177-4890-a2fc-c9eb4163e3dd None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:34.826462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:34.860662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:34.861279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:34.861565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:34.861850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:34.862463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:34.864190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING py.warnings [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:34.864190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: warnings.warn(msg) Aug 10 12:27:34.864190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: Aug 10 12:27:34.865409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:34.920854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 743/1497] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:27:34 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:27:34.925243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:34.926816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:34.959490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c619568-309c-49f1-a1ec-0ff681627b0f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:35.051994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:35.052595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:35.052870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:35.053139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:35.053790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:35.054805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:35.054805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:27:35.054805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:27:35.056013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:35.098534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 755/1498] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:27:34 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:27:35.102912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:35.104409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:35.165559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe169a-1cd6-448e-94ac-b579f31d474b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:35.246539 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:35.257723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:35.258588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:35.258874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:35.259139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:35.259772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:35.260781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING py.warnings [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:35.260781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: warnings.warn(msg) Aug 10 12:27:35.260781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: Aug 10 12:27:35.261687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:35.274643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 744/1499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:27:35.278702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:35.280904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:35.292296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4bfe4-40cf-49da-9d3e-c3cad86dab60 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:35.317619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:35.387527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57937b47-4354-47f4-a696-6eb7cb8f1505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:35.388061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57937b47-4354-47f4-a696-6eb7cb8f1505 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:35.388340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57937b47-4354-47f4-a696-6eb7cb8f1505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:35.388610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57937b47-4354-47f4-a696-6eb7cb8f1505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:35.389781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57937b47-4354-47f4-a696-6eb7cb8f1505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:35.390787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-57937b47-4354-47f4-a696-6eb7cb8f1505 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:35.390787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:27:35.390787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:27:35.391678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57937b47-4354-47f4-a696-6eb7cb8f1505 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:35.403741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 756/1500] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:27:35.408440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:35.409972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:35.442960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f4679-1ef8-4139-bff6-0e9c92e860b2 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:35.533900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e46d3-af6a-4689-9e40-49b201775304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:35.534539 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e46d3-af6a-4689-9e40-49b201775304 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:35.534811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e46d3-af6a-4689-9e40-49b201775304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:35.535077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e46d3-af6a-4689-9e40-49b201775304 admin admin] PATH_INFO: `/v3/projects/87fecb09a6204f13b6c910e4c5eb135f/users/2b916e8e74914b8e99c0a776b197ddbf/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:35.556303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e46d3-af6a-4689-9e40-49b201775304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fecb09a6204f13b6c910e4c5eb135f, user_id=2b916e8e74914b8e99c0a776b197ddbf, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:35.557371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING py.warnings [None req-8d6e46d3-af6a-4689-9e40-49b201775304 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:35.557371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: warnings.warn(msg) Aug 10 12:27:35.557371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: Aug 10 12:27:35.559294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e46d3-af6a-4689-9e40-49b201775304 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:35.584685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 745/1501] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:27:35 2022] PUT /identity/v3/projects/87fecb09a6204f13b6c910e4c5eb135f/users/2b916e8e74914b8e99c0a776b197ddbf/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:27:35.597337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba56b9-685d-42c9-b880-2ce95cd165a1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:35.597337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba56b9-685d-42c9-b880-2ce95cd165a1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:35.597552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba56b9-685d-42c9-b880-2ce95cd165a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:35.654550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-51ba56b9-685d-42c9-b880-2ce95cd165a1 None None] MFA Rules not processed for user `2b916e8e74914b8e99c0a776b197ddbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:35.655684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-51ba56b9-685d-42c9-b880-2ce95cd165a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:35.731987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 757/1502] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:27:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:27:35.745095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c85b2-446d-4b4c-b334-1c797620976a None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:35.745500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c85b2-446d-4b4c-b334-1c797620976a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:35.745780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c85b2-446d-4b4c-b334-1c797620976a None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:35.747113 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 746/1503] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 10 12:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:27:35.752444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206be4d-4ad4-442b-bb73-306cdf8246bd None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:35.752730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206be4d-4ad4-442b-bb73-306cdf8246bd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:35.753006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206be4d-4ad4-442b-bb73-306cdf8246bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:35.783694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:35.828780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-4206be4d-4ad4-442b-bb73-306cdf8246bd None None] MFA Rules not processed for user `89d07f733a6c44858ba396c933e370e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:35.830156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4206be4d-4ad4-442b-bb73-306cdf8246bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:35.904931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 758/1504] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 10 12:27:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:27:35.912681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c85b2-446d-4b4c-b334-1c797620976a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:35.914248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2c0c85b2-446d-4b4c-b334-1c797620976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:35.964281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:36.018296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c85b2-446d-4b4c-b334-1c797620976a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:36.093175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:36.093793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:36.094070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:36.094338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:36.095912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:36.192970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:36.194426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:36.197889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:36.212772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 747/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:27:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:27:36.257438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd79e6d-9416-4230-b8f7-e54602c2d8b1 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:36.257724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd79e6d-9416-4230-b8f7-e54602c2d8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:36.258025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd79e6d-9416-4230-b8f7-e54602c2d8b1 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:36.259308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 759/1506] 10.222.0.65 () {52 vars in 830 bytes} [Wed Aug 10 12:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:27:36.262972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:36.264237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:36.363872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b51fe-2c45-4b78-b80f-e9142096d577 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:36.455819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:36.456375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:36.456652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:36.456923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] PATH_INFO: `/v3/users/2b916e8e74914b8e99c0a776b197ddbf` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:36.468715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b916e8e74914b8e99c0a776b197ddbf)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:36.469764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING py.warnings [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:36.469764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: warnings.warn(msg) Aug 10 12:27:36.469764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: Aug 10 12:27:36.471084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:36.480863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:36.537608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b916e8e74914b8e99c0a776b197ddbf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:36.551521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b916e8e74914b8e99c0a776b197ddbf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:36.560531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b916e8e74914b8e99c0a776b197ddbf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:36.561577 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b916e8e74914b8e99c0a776b197ddbf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:36.562002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b916e8e74914b8e99c0a776b197ddbf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:36.590584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 748/1507] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:27:36 2022] DELETE /identity/v3/users/2b916e8e74914b8e99c0a776b197ddbf => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:27:36.595133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:36.595600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:36.595756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:36.651849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:27:36.653040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:36.726008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:36.739860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 760/1508] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:27:36.750010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:36.751216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:36.850727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1672671-b64e-4b79-be4d-018bc23b3aee admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:36.904787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:36.925428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c27ef1-7599-45ef-a9ff-36a0c96bf3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:36.926036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c27ef1-7599-45ef-a9ff-36a0c96bf3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:36.926312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c27ef1-7599-45ef-a9ff-36a0c96bf3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:36.926582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c27ef1-7599-45ef-a9ff-36a0c96bf3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:36.928147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-02c27ef1-7599-45ef-a9ff-36a0c96bf3f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:37.033565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c27ef1-7599-45ef-a9ff-36a0c96bf3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:37.034921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c27ef1-7599-45ef-a9ff-36a0c96bf3f9 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:37.053251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 749/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:27:37.416933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:27:37.418195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:27:37.429382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb5bed-1356-4f8a-9b35-dc994dfa062f None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:27:37.534771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:27:37.535405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:27:37.535685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:27:37.535956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] PATH_INFO: `/v3/projects/87fecb09a6204f13b6c910e4c5eb135f` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:27:37.545934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] RBAC: Authorizing `identity:delete_project(project_id=87fecb09a6204f13b6c910e4c5eb135f)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:27:37.546900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:27:37.546900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:27:37.546900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:27:37.548109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:27:37.605570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87fecb09a6204f13b6c910e4c5eb135f', 'request_id': 'req-9f57388d-3b4c-4743-ba41-af2e13cbaeab'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:37.611767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87fecb09a6204f13b6c910e4c5eb135f', 'request_id': 'req-9f57388d-3b4c-4743-ba41-af2e13cbaeab'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:37.621938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87fecb09a6204f13b6c910e4c5eb135f', 'request_id': 'req-9f57388d-3b4c-4743-ba41-af2e13cbaeab'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:37.651893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] The token cache is being invalidate because project 87fecb09a6204f13b6c910e4c5eb135f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:27:37.652205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:37.652710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9f57388d-3b4c-4743-ba41-af2e13cbaeab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:27:37.654009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 761/1510] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:27:37 2022] DELETE /identity/v3/projects/87fecb09a6204f13b6c910e4c5eb135f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:27:47.051031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=62 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:27:47.649926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:29:10.418285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f8ac-c053-46bc-880d-8233717ab5c9 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:10.418744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f8ac-c053-46bc-880d-8233717ab5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:10.418855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f8ac-c053-46bc-880d-8233717ab5c9 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:10.420222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:10.425711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58efe6f-3cc8-49e7-96ef-858c37a9a3e9 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:10.426133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58efe6f-3cc8-49e7-96ef-858c37a9a3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:10.426260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58efe6f-3cc8-49e7-96ef-858c37a9a3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:10.506592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c58efe6f-3cc8-49e7-96ef-858c37a9a3e9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:10.507784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c58efe6f-3cc8-49e7-96ef-858c37a9a3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:10.607790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:10.759892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f8ac-c053-46bc-880d-8233717ab5c9 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:10.762154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f087f8ac-c053-46bc-880d-8233717ab5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:10.879714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f8ac-c053-46bc-880d-8233717ab5c9 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:10.885157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454d4bb-14ce-4340-acf3-77288b3ac507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:10.885851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d4bb-14ce-4340-acf3-77288b3ac507 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:10.886136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d4bb-14ce-4340-acf3-77288b3ac507 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:10.886412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d4bb-14ce-4340-acf3-77288b3ac507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:10.894927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454d4bb-14ce-4340-acf3-77288b3ac507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:10.896528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454d4bb-14ce-4340-acf3-77288b3ac507 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:10.905867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 751/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:11.727085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872a700-387d-46a3-9eaf-6184d95599d8 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:11.727651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872a700-387d-46a3-9eaf-6184d95599d8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:11.727651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872a700-387d-46a3-9eaf-6184d95599d8 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:11.728985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:11.734569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b750c58-4681-418f-9a82-d6291e945f6c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:11.734971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b750c58-4681-418f-9a82-d6291e945f6c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:11.735144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b750c58-4681-418f-9a82-d6291e945f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:11.754137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5b750c58-4681-418f-9a82-d6291e945f6c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:11.755338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5b750c58-4681-418f-9a82-d6291e945f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:11.763098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:11.913893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872a700-387d-46a3-9eaf-6184d95599d8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:11.927939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872a700-387d-46a3-9eaf-6184d95599d8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:11.933336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e0adb-8d1c-46cf-9731-d01fc16ebffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:11.933923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0adb-8d1c-46cf-9731-d01fc16ebffe service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:11.934198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0adb-8d1c-46cf-9731-d01fc16ebffe service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:11.934465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0adb-8d1c-46cf-9731-d01fc16ebffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:11.942441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e0adb-8d1c-46cf-9731-d01fc16ebffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:11.944026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e0adb-8d1c-46cf-9731-d01fc16ebffe service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:11.953741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 764/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:13.106310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c35610-5643-4b59-96e0-a82ae87dba54 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:13.106886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c35610-5643-4b59-96e0-a82ae87dba54 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:13.106886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c35610-5643-4b59-96e0-a82ae87dba54 None None] PATH_INFO: `/v3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:13.108834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 753/1517] 10.222.0.64 () {54 vars in 925 bytes} [Wed Aug 10 12:29:13 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:29:13.114994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe7809-a4d0-4781-9852-a2a1175791a6 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:13.115419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe7809-a4d0-4781-9852-a2a1175791a6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:13.115622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe7809-a4d0-4781-9852-a2a1175791a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:13.173274 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-85fe7809-a4d0-4781-9852-a2a1175791a6 None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:13.174505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-85fe7809-a4d0-4781-9852-a2a1175791a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:13.190884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:29:13.245034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 765/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Aug 10 12:29:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:21.750100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=56 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:29:22.853427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b7dc4-75b2-4d15-af95-feabf2e63196 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:22.854036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b7dc4-75b2-4d15-af95-feabf2e63196 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:22.854036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b7dc4-75b2-4d15-af95-feabf2e63196 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:22.855383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 754/1519] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 10 12:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:29:22.860176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1ba08-4ecd-481d-ad7d-0ce04dc849f2 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:22.860589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1ba08-4ecd-481d-ad7d-0ce04dc849f2 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:22.860707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1ba08-4ecd-481d-ad7d-0ce04dc849f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:22.880090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-efe1ba08-4ecd-481d-ad7d-0ce04dc849f2 None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:22.881297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-efe1ba08-4ecd-481d-ad7d-0ce04dc849f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:22.890092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 766/1520] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 10 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:22.898100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b7dc4-75b2-4d15-af95-feabf2e63196 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:22.911350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b7dc4-75b2-4d15-af95-feabf2e63196 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:22.916259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c40928-561e-4f7d-b709-51a7271614f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:22.916854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40928-561e-4f7d-b709-51a7271614f5 service placement] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:22.917134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40928-561e-4f7d-b709-51a7271614f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:22.917441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40928-561e-4f7d-b709-51a7271614f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:22.925791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c40928-561e-4f7d-b709-51a7271614f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:22.927636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c40928-561e-4f7d-b709-51a7271614f5 service placement] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:22.938196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 755/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:23.174684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470881-d240-40c7-8fc0-68197d3a2041 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:23.174973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470881-d240-40c7-8fc0-68197d3a2041 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:23.175252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470881-d240-40c7-8fc0-68197d3a2041 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:23.176629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:23.181418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c4c89-73ff-4138-ba19-349fe25ba2e0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:23.181708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c4c89-73ff-4138-ba19-349fe25ba2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:23.181984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c4c89-73ff-4138-ba19-349fe25ba2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:23.200917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b84c4c89-73ff-4138-ba19-349fe25ba2e0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:23.202144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b84c4c89-73ff-4138-ba19-349fe25ba2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:23.210595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:23.363948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9470881-d240-40c7-8fc0-68197d3a2041 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:23.378584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9470881-d240-40c7-8fc0-68197d3a2041 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:23.383921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e0496-5d7f-4a62-901c-a57c64665711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:23.384514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e0496-5d7f-4a62-901c-a57c64665711 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:23.384794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e0496-5d7f-4a62-901c-a57c64665711 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:23.385065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e0496-5d7f-4a62-901c-a57c64665711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:23.393630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e0496-5d7f-4a62-901c-a57c64665711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:23.395213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e0496-5d7f-4a62-901c-a57c64665711 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:23.404376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 768/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:24.219264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc7faa-7e87-4be5-8086-8d58ac175295 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:24.219817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc7faa-7e87-4be5-8086-8d58ac175295 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:24.219817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc7faa-7e87-4be5-8086-8d58ac175295 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:24.221165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 12:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:24.226212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb7890-21b3-494f-9724-85207625f9ba None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:24.226603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb7890-21b3-494f-9724-85207625f9ba None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:24.226760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb7890-21b3-494f-9724-85207625f9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:24.246073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d2bb7890-21b3-494f-9724-85207625f9ba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:24.247251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d2bb7890-21b3-494f-9724-85207625f9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:24.254946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:24.407806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc7faa-7e87-4be5-8086-8d58ac175295 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:24.422962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc7faa-7e87-4be5-8086-8d58ac175295 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:24.428574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13fbff-0278-41d8-89c1-94a5ab973f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:24.429180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13fbff-0278-41d8-89c1-94a5ab973f29 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:24.429492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13fbff-0278-41d8-89c1-94a5ab973f29 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:24.429767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13fbff-0278-41d8-89c1-94a5ab973f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:24.438638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13fbff-0278-41d8-89c1-94a5ab973f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:24.440237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13fbff-0278-41d8-89c1-94a5ab973f29 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:24.449840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 758/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:28.402447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1251c-7446-4d69-8f9d-968659ae895b None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:28.403007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1251c-7446-4d69-8f9d-968659ae895b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:28.403007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1251c-7446-4d69-8f9d-968659ae895b None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:28.404299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:28.409480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede1189-4ece-4431-b530-b1190e177249 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:28.409895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede1189-4ece-4431-b530-b1190e177249 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:28.410034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede1189-4ece-4431-b530-b1190e177249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:28.430796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6ede1189-4ece-4431-b530-b1190e177249 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:28.432000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6ede1189-4ece-4431-b530-b1190e177249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:28.440515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:28.512723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1251c-7446-4d69-8f9d-968659ae895b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:28.514433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d4d1251c-7446-4d69-8f9d-968659ae895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:28.594235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1251c-7446-4d69-8f9d-968659ae895b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:28.597999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e36e4-1a74-4e2d-909e-a414188818b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:28.598798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e36e4-1a74-4e2d-909e-a414188818b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:28.599079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e36e4-1a74-4e2d-909e-a414188818b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:28.599351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e36e4-1a74-4e2d-909e-a414188818b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:28.608448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e36e4-1a74-4e2d-909e-a414188818b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:28.610527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e36e4-1a74-4e2d-909e-a414188818b7 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:28.619897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 771/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:30.620402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1f9d6-52d4-4413-ac74-c7663b3a851d None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:30.620899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1f9d6-52d4-4413-ac74-c7663b3a851d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:30.621006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1f9d6-52d4-4413-ac74-c7663b3a851d None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:30.622343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:30.627634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0ca92-ab94-4063-aa94-1515e46f60d5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:30.628055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0ca92-ab94-4063-aa94-1515e46f60d5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:30.628159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0ca92-ab94-4063-aa94-1515e46f60d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:30.647775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-32f0ca92-ab94-4063-aa94-1515e46f60d5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:30.648961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-32f0ca92-ab94-4063-aa94-1515e46f60d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:30.657144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:30.691822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1f9d6-52d4-4413-ac74-c7663b3a851d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:30.703259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1f9d6-52d4-4413-ac74-c7663b3a851d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:30.706983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daa34b-6bf4-459f-85ec-0e50c3ea4773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:30.707568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daa34b-6bf4-459f-85ec-0e50c3ea4773 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:30.707842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daa34b-6bf4-459f-85ec-0e50c3ea4773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:30.708112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daa34b-6bf4-459f-85ec-0e50c3ea4773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:30.716634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daa34b-6bf4-459f-85ec-0e50c3ea4773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:30.718115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daa34b-6bf4-459f-85ec-0e50c3ea4773 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:30.727982 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 761/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:34.307666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34be488-afb5-4648-afb9-a590b08b664c None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:34.308226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34be488-afb5-4648-afb9-a590b08b664c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:34.308226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34be488-afb5-4648-afb9-a590b08b664c None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:34.309581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:34.314750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05a963-2c7d-4aed-8471-7972288356f9 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:34.315160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05a963-2c7d-4aed-8471-7972288356f9 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:34.315303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05a963-2c7d-4aed-8471-7972288356f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:34.335713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3a05a963-2c7d-4aed-8471-7972288356f9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:34.337584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3a05a963-2c7d-4aed-8471-7972288356f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:34.346733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:34.653523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34be488-afb5-4648-afb9-a590b08b664c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:34.655205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a34be488-afb5-4648-afb9-a590b08b664c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:34.718207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:29:34.732087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34be488-afb5-4648-afb9-a590b08b664c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:34.735659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6edcf-4783-482b-a08d-19e18b244f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:34.736255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6edcf-4783-482b-a08d-19e18b244f67 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:34.736530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6edcf-4783-482b-a08d-19e18b244f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:34.736799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6edcf-4783-482b-a08d-19e18b244f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:34.745479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6edcf-4783-482b-a08d-19e18b244f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:34.746908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6edcf-4783-482b-a08d-19e18b244f67 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:34.756498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 774/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:29:36.738099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6fc1-a22e-4edf-9148-661a40df88b7 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:36.738615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6fc1-a22e-4edf-9148-661a40df88b7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:36.738615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6fc1-a22e-4edf-9148-661a40df88b7 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:36.739927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 12:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:29:36.745116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f639-324e-4e5f-8037-ddfe72ffa019 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:36.745527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f639-324e-4e5f-8037-ddfe72ffa019 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:36.745698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f639-324e-4e5f-8037-ddfe72ffa019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:36.766066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5501f639-324e-4e5f-8037-ddfe72ffa019 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:29:36.767281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5501f639-324e-4e5f-8037-ddfe72ffa019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:36.775172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 12:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:29:37.079748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6fc1-a22e-4edf-9148-661a40df88b7 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:29:37.081472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-402b6fc1-a22e-4edf-9148-661a40df88b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:29:37.094912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6fc1-a22e-4edf-9148-661a40df88b7 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:29:37.098668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479caec-f184-41ae-95ae-0923b044be3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:29:37.099260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479caec-f184-41ae-95ae-0923b044be3f service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:29:37.099542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479caec-f184-41ae-95ae-0923b044be3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:29:37.099818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479caec-f184-41ae-95ae-0923b044be3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:29:37.110047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479caec-f184-41ae-95ae-0923b044be3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:29:37.111507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479caec-f184-41ae-95ae-0923b044be3f service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:29:37.121174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 764/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:29:46.761161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=11 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:29:47.117011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=33 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:30:59.371436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068e466-f180-450a-9f4b-55bc638eb540 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.375224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068e466-f180-450a-9f4b-55bc638eb540 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.375224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068e466-f180-450a-9f4b-55bc638eb540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.395593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6256-7c86-4d2e-b41b-1d93e98d0f52 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.396666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6256-7c86-4d2e-b41b-1d93e98d0f52 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.397185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6256-7c86-4d2e-b41b-1d93e98d0f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.401604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5068e466-f180-450a-9f4b-55bc638eb540 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.402876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5068e466-f180-450a-9f4b-55bc638eb540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.415265 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 776/1540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.427096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-08fc6256-7c86-4d2e-b41b-1d93e98d0f52 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.428325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-08fc6256-7c86-4d2e-b41b-1d93e98d0f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.444322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 765/1541] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.446405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d4c9-18c6-4ecf-a755-96a057fa33f8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.446930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d4c9-18c6-4ecf-a755-96a057fa33f8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.447416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d4c9-18c6-4ecf-a755-96a057fa33f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.453807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be9c45-dad5-4e20-9ce4-dfa30e2c5481 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.454095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be9c45-dad5-4e20-9ce4-dfa30e2c5481 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.454368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be9c45-dad5-4e20-9ce4-dfa30e2c5481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.475892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2684d4c9-18c6-4ecf-a755-96a057fa33f8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.477541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2684d4c9-18c6-4ecf-a755-96a057fa33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.482747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d1be9c45-dad5-4e20-9ce4-dfa30e2c5481 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.484217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d1be9c45-dad5-4e20-9ce4-dfa30e2c5481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.493759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 777/1542] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.495248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5ab68-487d-45df-91aa-08c713df6ce9 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.495797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5ab68-487d-45df-91aa-08c713df6ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.496297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5ab68-487d-45df-91aa-08c713df6ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.497623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 766/1543] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.503156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be9c45-dad5-4e20-9ce4-dfa30e2c5481 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.517121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be9c45-dad5-4e20-9ce4-dfa30e2c5481 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.525209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.525856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.526138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.526415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.527075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:30:59.527383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.528409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING py.warnings [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:30:59.528409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: warnings.warn(msg) Aug 10 12:30:59.528409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: Aug 10 12:30:59.529331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:30:59.531483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e6e5ab68-487d-45df-91aa-08c713df6ce9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.532911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e6e5ab68-487d-45df-91aa-08c713df6ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.542017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 767/1544] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:30:59.543088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.555281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe31dc4-736d-4c97-a6fc-aa62600a575a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.556916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 778/1545] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.558438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8adb-acdc-4b4c-a0b4-0f1f74132fbe None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.559462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8adb-acdc-4b4c-a0b4-0f1f74132fbe None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.560243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8adb-acdc-4b4c-a0b4-0f1f74132fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.590724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.594830 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.595195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.595483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.596176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:30:59.596539 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.601493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:30:59.614769 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-40bd8adb-acdc-4b4c-a0b4-0f1f74132fbe None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.616313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-40bd8adb-acdc-4b4c-a0b4-0f1f74132fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.622607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 768/1546] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:30:59.622607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.632656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8bd99-0a4a-4c21-bace-9cb6874eef36 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.641830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 779/1547] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.642731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.643199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c3e5-c514-48e3-b2c1-ba4eb4df4f4f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.643398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.643520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c3e5-c514-48e3-b2c1-ba4eb4df4f4f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.643701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.643796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71c3e5-c514-48e3-b2c1-ba4eb4df4f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.643942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.644556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.646515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:30:59.675314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-cc71c3e5-c514-48e3-b2c1-ba4eb4df4f4f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.676570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-cc71c3e5-c514-48e3-b2c1-ba4eb4df4f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.692033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 780/1548] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.693618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20044b09-a7cb-4de1-8ad0-812f7f12635e None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.693907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20044b09-a7cb-4de1-8ad0-812f7f12635e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.694179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20044b09-a7cb-4de1-8ad0-812f7f12635e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.704616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 769/1549] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/projects => generated 405 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:30:59.705767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.714784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-20044b09-a7cb-4de1-8ad0-812f7f12635e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.715970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-20044b09-a7cb-4de1-8ad0-812f7f12635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.769223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf76d0-a8d5-4af1-ad10-8871726ba829 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.774262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88845440-cd0f-483c-85fd-af05e5c0c8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.774682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 781/1550] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.775218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88845440-cd0f-483c-85fd-af05e5c0c8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.775443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88845440-cd0f-483c-85fd-af05e5c0c8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.775714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88845440-cd0f-483c-85fd-af05e5c0c8d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.775914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04b888-acb1-4cbb-b4e0-21f31424123b None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.776203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04b888-acb1-4cbb-b4e0-21f31424123b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.776376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88845440-cd0f-483c-85fd-af05e5c0c8d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.776489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04b888-acb1-4cbb-b4e0-21f31424123b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.778308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88845440-cd0f-483c-85fd-af05e5c0c8d8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:30:59.795772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fa04b888-acb1-4cbb-b4e0-21f31424123b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.797807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fa04b888-acb1-4cbb-b4e0-21f31424123b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.809094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 770/1551] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:30:59.809303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 782/1552] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.809420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa44db-5fbf-4e68-8989-e69ee5e0c029 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.809420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa44db-5fbf-4e68-8989-e69ee5e0c029 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.809420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa44db-5fbf-4e68-8989-e69ee5e0c029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.809661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29a15-a6ba-4605-8c96-3a4a122fa4d6 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.809661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29a15-a6ba-4605-8c96-3a4a122fa4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.809661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29a15-a6ba-4605-8c96-3a4a122fa4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.827962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2caa44db-5fbf-4e68-8989-e69ee5e0c029 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.829157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2caa44db-5fbf-4e68-8989-e69ee5e0c029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.829857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-07c29a15-a6ba-4605-8c96-3a4a122fa4d6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.831035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-07c29a15-a6ba-4605-8c96-3a4a122fa4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.888214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 771/1553] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.889405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a106f93-6d5e-47e1-8dce-4b39ca0568c6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.889721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a106f93-6d5e-47e1-8dce-4b39ca0568c6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.889992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a106f93-6d5e-47e1-8dce-4b39ca0568c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.892210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 783/1554] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.893206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c29a15-a6ba-4605-8c96-3a4a122fa4d6 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.902787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c29a15-a6ba-4605-8c96-3a4a122fa4d6 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.907389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.908146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.908424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.908690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.909370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:30:59.909672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.911296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:30:59.916228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9a106f93-6d5e-47e1-8dce-4b39ca0568c6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.917453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9a106f93-6d5e-47e1-8dce-4b39ca0568c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.923324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 784/1555] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:30:59.924133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.926411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 772/1556] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.930533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef177d2-a7cd-4e10-9881-70e93e101979 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.930815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef177d2-a7cd-4e10-9881-70e93e101979 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.931087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef177d2-a7cd-4e10-9881-70e93e101979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.939093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba604b-786c-4422-9f98-225ca9940200 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.943808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.944409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.944687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.944958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.945986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.947571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:30:59.950773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5ef177d2-a7cd-4e10-9881-70e93e101979 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:30:59.951968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5ef177d2-a7cd-4e10-9881-70e93e101979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:30:59.961427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 773/1557] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:30:59.962153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef177d2-a7cd-4e10-9881-70e93e101979 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.966927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:30:59.970127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 785/1558] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:30:59.971155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:30:59.975636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef177d2-a7cd-4e10-9881-70e93e101979 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.980161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8e323-3044-49f4-a0aa-be60db43bc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.980754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8e323-3044-49f4-a0aa-be60db43bc21 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.980879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bd419-151a-4024-9fbd-b8457b3ce8ac admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:30:59.981034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8e323-3044-49f4-a0aa-be60db43bc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.981342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8e323-3044-49f4-a0aa-be60db43bc21 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.981965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8e323-3044-49f4-a0aa-be60db43bc21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.983021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING py.warnings [None req-fda8e323-3044-49f4-a0aa-be60db43bc21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:30:59.983021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: warnings.warn(msg) Aug 10 12:30:59.983021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: Aug 10 12:30:59.984722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8e323-3044-49f4-a0aa-be60db43bc21 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:30:59.985816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b00fb-c312-4cad-a929-15f18b73bfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:30:59.986396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b00fb-c312-4cad-a929-15f18b73bfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:30:59.986672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b00fb-c312-4cad-a929-15f18b73bfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:30:59.986944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b00fb-c312-4cad-a929-15f18b73bfa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:30:59.987582 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b00fb-c312-4cad-a929-15f18b73bfa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:30:59.987885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b00fb-c312-4cad-a929-15f18b73bfa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:30:59.989484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b00fb-c312-4cad-a929-15f18b73bfa9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.001823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 786/1559] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:00.003026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ac971-6316-460f-bfb1-c323a2e3b128 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.003312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ac971-6316-460f-bfb1-c323a2e3b128 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.003617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ac971-6316-460f-bfb1-c323a2e3b128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.007368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 774/1560] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:30:59 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.008582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d79c7-4285-468f-b75f-a1ec4797ddde None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.008869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d79c7-4285-468f-b75f-a1ec4797ddde None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.009138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d79c7-4285-468f-b75f-a1ec4797ddde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.026165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-f85ac971-6316-460f-bfb1-c323a2e3b128 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:00.027377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f85ac971-6316-460f-bfb1-c323a2e3b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:00.027690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6d6d79c7-4285-468f-b75f-a1ec4797ddde None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:00.028874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6d6d79c7-4285-468f-b75f-a1ec4797ddde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:00.035810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 787/1561] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:00.036541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ac971-6316-460f-bfb1-c323a2e3b128 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.036967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 775/1562] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:00.038142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafacc-4f28-4c18-b0a5-f030048891c0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.038425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafacc-4f28-4c18-b0a5-f030048891c0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.038695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafacc-4f28-4c18-b0a5-f030048891c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.047718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ac971-6316-460f-bfb1-c323a2e3b128 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.052310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.052910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.053185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.053488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.054104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.055120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:31:00.055120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:31:00.055120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:31:00.056314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.059588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-cfaafacc-4f28-4c18-b0a5-f030048891c0 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:00.060774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cfaafacc-4f28-4c18-b0a5-f030048891c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:00.065742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:00.069030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 776/1563] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:00.069859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaafacc-4f28-4c18-b0a5-f030048891c0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.079455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaafacc-4f28-4c18-b0a5-f030048891c0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.084500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.085130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.085667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.085958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.086610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:00.086944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.090172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 788/1564] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.090172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.090383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.101856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 777/1565] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:00.102652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.118485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97769d5d-cceb-4020-b23b-3485874f0df5 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.148863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14ab1c-6e65-40df-9f05-41f380d88d64 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.154155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.154763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.155041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.155840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.156482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.158170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.168025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 789/1566] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.168627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.177098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.177725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.178004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.178269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.178369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a9814-3012-4ae5-a3b1-c01cf8263804 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.178888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.180776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.182940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.183523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.183803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.184104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.184728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.186357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.196106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 790/1567] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.197736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.207807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 778/1568] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.208224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d0c48-0cb8-4e5e-88f2-d2b49784484d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.209045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.265941 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.266546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.266823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.267092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.267728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:00.268030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.269624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c55c57-5942-40ad-97c3-fd66d5213412 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.269747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.274367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.275078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.275364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.275638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.276761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:00.277073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.279287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.281798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 791/1569] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:00.282583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.291785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032a2c6-20af-4f5e-b14f-fd577776a8d2 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.296991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.297610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.297890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.298163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.298795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:00.299267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.299387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 779/1570] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:00.299921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.300690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.310659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81e458-d166-4266-808e-85e6aaa8bebc admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.316906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 792/1571] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:00.316906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.319393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.320001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.320273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.320556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.321178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.323232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.328709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fad32-5f00-464c-b05e-07388db1336d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.333531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.334123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.334400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.334670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.335308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.337292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.349465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 780/1572] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.349465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.356410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bed88-3434-4048-b62c-059f43bd241f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.362199 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.362789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.363067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.363336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.363965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.365583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.368314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 793/1573] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.370166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.379901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 781/1574] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.379901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.387077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a83e2-46d3-47e4-aa32-c790f65d9e62 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.391941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:00.403915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:00.433913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9f21e-8a6c-4a4d-b029-3775a4be0d60 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.438607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.439207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.439482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.439749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.440380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.442080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.445522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.446129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.446408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.446677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.447303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.449220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.456129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 794/1575] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.456129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.462514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c22061-ce81-4794-996c-e1d0df74ce6d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.467668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.468961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.469240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.469535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.470149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.472034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.474785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 782/1576] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.475637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.485160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5336911-0486-498b-b60c-fa5f22d8c616 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.489965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.490547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.490833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.491113 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.491797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.494453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.501893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 795/1577] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.502798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.522718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 783/1578] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.523508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.564873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2ff4f-84e7-463c-ba13-0a6cfe9e6df5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.585297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb5c61-d156-4bbf-a662-22a70f979c78 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.589798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.590401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.590683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.590954 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.591632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.593875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.604444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 784/1579] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.605087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.614544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36bdd1-4e03-496b-8c56-fecd03f3abfd admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.619108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.619685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.619966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.620240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.620849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.622779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.626592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.627200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.627595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.627867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.628479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.630417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.645093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 785/1580] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.646204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.656325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cedc08-1d7d-4ed3-9eed-3fce02f10d0e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.661444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.662081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.662359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.662627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] PATH_INFO: `/v3/projects/8f8ff504b0b945d9a1bcd73152eda5eb/users/6a3075c65d63411bb3da76276b28355c/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.667368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 796/1581] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.668365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.677850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8ff504b0b945d9a1bcd73152eda5eb, user_id=6a3075c65d63411bb3da76276b28355c, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.680587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.692912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 786/1582] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:00 2022] PUT /identity/v3/projects/8f8ff504b0b945d9a1bcd73152eda5eb/users/6a3075c65d63411bb3da76276b28355c/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:00.693808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.720993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:00.727755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:00.742982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d533-708f-4e95-9734-cb2d6513f93e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.757224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64912b29-2b1a-4a25-b1f0-a882ce499b06 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.763553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.764179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.764458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.765218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.765931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.767985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.777763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 787/1583] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.778626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.788684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499847e-8963-4fda-9f82-78001f5e72fa admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.793164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.793766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.794045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.794313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.794912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.796845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.798348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.799003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.799282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.799556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] PATH_INFO: `/v3/projects/28eda660c33645f8a4428987084ab934/users/94473332fd86422a9f1a1eed466f5a5a/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.816696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28eda660c33645f8a4428987084ab934, user_id=94473332fd86422a9f1a1eed466f5a5a, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.817764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:31:00.817764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:31:00.817764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:31:00.819707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.820118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 788/1584] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.820990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.833773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8c407-cfe2-4fd0-8c01-52e999132384 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.834987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 797/1585] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:00 2022] PUT /identity/v3/projects/28eda660c33645f8a4428987084ab934/users/94473332fd86422a9f1a1eed466f5a5a/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:00.835794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.892338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.892946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.893223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.893528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.894156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.896091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.898948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4ad09-ec07-431d-9b3f-da7eaef9e7d5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.903639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.904234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.904509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.904963 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.905463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.907113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.917802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 798/1586] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.921425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 789/1587] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.921564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.921688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.929024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ab835-11f4-4b9a-95a8-0fd4096c32ad admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.930619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83884c-5d37-451a-9af6-453478bf6130 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.934039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.934657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.934937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.935044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655b34-6713-4ace-80d0-587097089daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.935262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.935595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655b34-6713-4ace-80d0-587097089daf admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.935869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655b34-6713-4ace-80d0-587097089daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.935976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.936136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655b34-6713-4ace-80d0-587097089daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.937259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655b34-6713-4ace-80d0-587097089daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.938512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.938876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655b34-6713-4ace-80d0-587097089daf admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.948529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 799/1588] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.949720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d132ed2-8382-4988-85e4-94cadf2b2ab1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.950157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d132ed2-8382-4988-85e4-94cadf2b2ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.950317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d132ed2-8382-4988-85e4-94cadf2b2ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.961568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 790/1589] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/users => generated 294 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:00.962566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:00.971760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48542a-8b4f-47b6-af1e-12eac1e39b32 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:00.976475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:00.977053 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:00.977388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:00.977658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:00.978285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:00.979862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:00.982128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9d132ed2-8382-4988-85e4-94cadf2b2ab1 None None] MFA Rules not processed for user `6a3075c65d63411bb3da76276b28355c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:00.983415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9d132ed2-8382-4988-85e4-94cadf2b2ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:00.989545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 791/1590] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:00.991121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.000956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d624756-d937-41e7-8e38-b2209dc6ccd5 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.005698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac3039-0f82-4f78-9609-cb7d309b6fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.006271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac3039-0f82-4f78-9609-cb7d309b6fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.006548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac3039-0f82-4f78-9609-cb7d309b6fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.006815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac3039-0f82-4f78-9609-cb7d309b6fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.007436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac3039-0f82-4f78-9609-cb7d309b6fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.009038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac3039-0f82-4f78-9609-cb7d309b6fc3 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.022878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 792/1591] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:01.022878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f99b-be31-41e6-96c8-cf8ac770ef2a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.022878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f99b-be31-41e6-96c8-cf8ac770ef2a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.022878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f99b-be31-41e6-96c8-cf8ac770ef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.029291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:01.040019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3eb5f99b-be31-41e6-96c8-cf8ac770ef2a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:01.041236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3eb5f99b-be31-41e6-96c8-cf8ac770ef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:01.048711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 793/1592] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:01.049480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f99b-be31-41e6-96c8-cf8ac770ef2a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.050495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 800/1593] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:01.051302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d132ed2-8382-4988-85e4-94cadf2b2ab1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.060667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f99b-be31-41e6-96c8-cf8ac770ef2a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.062334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d132ed2-8382-4988-85e4-94cadf2b2ab1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.065760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.066439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.066722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.066848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.067017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] PATH_INFO: `/v3/projects/33d4ac4ff35f4130ac74c595ac69f0bc/users/c215a6f4372240cb914577c37e3aee30/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.067365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.067640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.067908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.068542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.070183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.080585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 801/1594] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:01.081497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.083057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d4ac4ff35f4130ac74c595ac69f0bc, user_id=c215a6f4372240cb914577c37e3aee30, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.085753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.091386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a00f9-d3e8-4bd3-a248-d96be10cf53c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.096661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.097248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.097557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.098094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.098721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.099031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 794/1595] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:01 2022] PUT /identity/v3/projects/33d4ac4ff35f4130ac74c595ac69f0bc/users/c215a6f4372240cb914577c37e3aee30/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:01.100312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.102165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.112120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 802/1596] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:01.112809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.122583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f3f4-8e4b-4d9e-bbfe-c6a8ae2e6f1b admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.135686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:01.167136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c295-0ac4-497e-805d-9e4159ac3cb5 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.172355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.172963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.173240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.173542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.174181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.175785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.182375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.183029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.183307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.183580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] PATH_INFO: `/v3/projects/a99cd651b5c8474fb336b307cbba38cd/users/9ea99f7a48134f2b8a36f5d67c22ad51/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.188618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 795/1597] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:01.188618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.196066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3f4f-bfc6-466f-9148-9e0155545f08 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.198877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99cd651b5c8474fb336b307cbba38cd, user_id=9ea99f7a48134f2b8a36f5d67c22ad51, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.200660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.201244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.201550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.201665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.201823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.202968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.204539 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.214649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 796/1598] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:01.215426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.217546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 803/1599] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:01 2022] PUT /identity/v3/projects/a99cd651b5c8474fb336b307cbba38cd/users/9ea99f7a48134f2b8a36f5d67c22ad51/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:01.217546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.217546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:01.229610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63445ce0-3145-4f08-874a-8052a009cc09 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.282346 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b1063-3c46-4bde-ae82-1067f7ea3f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.282687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ddbee-a6fa-4ae5-ac3d-a06cca82706b admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.282948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1063-3c46-4bde-ae82-1067f7ea3f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.283230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1063-3c46-4bde-ae82-1067f7ea3f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.283500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1063-3c46-4bde-ae82-1067f7ea3f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.284114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b1063-3c46-4bde-ae82-1067f7ea3f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.286163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b1063-3c46-4bde-ae82-1067f7ea3f8f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.286395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.287030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.287308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.287580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.296845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.298804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.308999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 804/1600] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:01.309839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.319090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 797/1601] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:01 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:01.319090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51909064-210a-4f9d-b294-fde3bbb71e72 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.319090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51909064-210a-4f9d-b294-fde3bbb71e72 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.319090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51909064-210a-4f9d-b294-fde3bbb71e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.323117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe15d7-d53d-41fa-a55f-0b11e0e0f9e5 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.350143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-51909064-210a-4f9d-b294-fde3bbb71e72 None None] MFA Rules not processed for user `c215a6f4372240cb914577c37e3aee30`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:01.351422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-51909064-210a-4f9d-b294-fde3bbb71e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:01.378026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:01.395706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.396955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.397403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.398058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.398740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.401585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.414418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 805/1602] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:01.418881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.421480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 798/1603] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:01.425979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51909064-210a-4f9d-b294-fde3bbb71e72 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.431456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32120583-1395-4333-a403-36aea4d07da4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.436928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c96c2-9679-490c-81c8-f8253e781c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.437690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c96c2-9679-490c-81c8-f8253e781c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.437917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c96c2-9679-490c-81c8-f8253e781c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.438445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c96c2-9679-490c-81c8-f8253e781c15 admin admin] PATH_INFO: `/v3/projects/0bbb137a895941b1876eebfc56c28425/users/9b8e4aad09814f8299a334639e34d948/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.439389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51909064-210a-4f9d-b294-fde3bbb71e72 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.445138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.446116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.446677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.446677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.447385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.449510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.455861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c96c2-9679-490c-81c8-f8253e781c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bbb137a895941b1876eebfc56c28425, user_id=9b8e4aad09814f8299a334639e34d948, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.459226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c96c2-9679-490c-81c8-f8253e781c15 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.461981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 799/1604] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:01.462926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.475273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e09eb1-4c87-4eab-8799-0b2e1c25a58c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.478813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 806/1605] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:01 2022] PUT /identity/v3/projects/0bbb137a895941b1876eebfc56c28425/users/9b8e4aad09814f8299a334639e34d948/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:01.482515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a823854-8203-478e-ba97-d79459b0bf21 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.482515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a823854-8203-478e-ba97-d79459b0bf21 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.482515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a823854-8203-478e-ba97-d79459b0bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.493482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:01.513401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-4a823854-8203-478e-ba97-d79459b0bf21 None None] MFA Rules not processed for user `9ea99f7a48134f2b8a36f5d67c22ad51`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:01.515268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4a823854-8203-478e-ba97-d79459b0bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:01.554410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.556538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.556538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.556538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] PATH_INFO: `/v3/projects/6db6bc4e043a4cde8d3c63cf84a1122f/users/57c9881ba9f94c86bae34ceb4ab30924/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.578403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db6bc4e043a4cde8d3c63cf84a1122f, user_id=57c9881ba9f94c86bae34ceb4ab30924, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.581550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.593567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 807/1606] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:01.594333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a823854-8203-478e-ba97-d79459b0bf21 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.603662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 800/1607] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:01 2022] PUT /identity/v3/projects/6db6bc4e043a4cde8d3c63cf84a1122f/users/57c9881ba9f94c86bae34ceb4ab30924/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:01.605039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.609849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a823854-8203-478e-ba97-d79459b0bf21 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.683539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.684122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.684410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.684686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.685345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.688046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.691377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ebc97-5b3a-4c61-bf96-e4605366a2d1 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.696101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.697373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.697666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.697948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] PATH_INFO: `/v3/projects/2301f937213e47c29686be4e8601ac32/users/2aa6d6d56f9f4618ae3883c9beb2e77b/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.710780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 808/1608] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:01 2022] POST /identity/v3/users => generated 308 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:01.711502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.720370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2301f937213e47c29686be4e8601ac32, user_id=2aa6d6d56f9f4618ae3883c9beb2e77b, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.723997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.739480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 801/1609] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:01 2022] PUT /identity/v3/projects/2301f937213e47c29686be4e8601ac32/users/2aa6d6d56f9f4618ae3883c9beb2e77b/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:01.740897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.763874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:01.781032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf28f9-f934-4479-aaf1-2c94f016f184 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.828378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cb85b-6f5e-471c-989c-aa62a33afae2 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:01.835780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d93923-aa42-4945-a059-05153c573ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.836696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93923-aa42-4945-a059-05153c573ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.837029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93923-aa42-4945-a059-05153c573ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.837448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93923-aa42-4945-a059-05153c573ca0 admin admin] PATH_INFO: `/v3/projects/4eb5c9a17d7341d3a3dd60ef2b7a381d/users/a78bd1aed509474b965592c478cb1c0c/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.858930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d93923-aa42-4945-a059-05153c573ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb5c9a17d7341d3a3dd60ef2b7a381d, user_id=a78bd1aed509474b965592c478cb1c0c, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.862375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d93923-aa42-4945-a059-05153c573ca0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.862730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102d11b-3202-427a-bf78-00962586d6bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:01.863369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d11b-3202-427a-bf78-00962586d6bd service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.863656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d11b-3202-427a-bf78-00962586d6bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.863930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102d11b-3202-427a-bf78-00962586d6bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.878290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 802/1610] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:01 2022] PUT /identity/v3/projects/4eb5c9a17d7341d3a3dd60ef2b7a381d/users/a78bd1aed509474b965592c478cb1c0c/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:01.879830 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70334ec-642d-4a76-9d6e-7a27f1d670a1 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.880204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70334ec-642d-4a76-9d6e-7a27f1d670a1 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.880688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70334ec-642d-4a76-9d6e-7a27f1d670a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.894367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:01.901099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b70334ec-642d-4a76-9d6e-7a27f1d670a1 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:01.902786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b70334ec-642d-4a76-9d6e-7a27f1d670a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:01.938502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102d11b-3202-427a-bf78-00962586d6bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:01.939868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102d11b-3202-427a-bf78-00962586d6bd service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:01.949021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 809/1611] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:01.950402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389df5ef-6f60-4ab7-9e7a-ea4b4b83de46 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.950696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389df5ef-6f60-4ab7-9e7a-ea4b4b83de46 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.950971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389df5ef-6f60-4ab7-9e7a-ea4b4b83de46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.978881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 803/1612] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:01.984139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faef2c2-1a2f-464a-ad52-dabd0169882d None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:01.984687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faef2c2-1a2f-464a-ad52-dabd0169882d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:01.985071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faef2c2-1a2f-464a-ad52-dabd0169882d None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:01.986881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 804/1613] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 10 12:31:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:01.989069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-389df5ef-6f60-4ab7-9e7a-ea4b4b83de46 None None] MFA Rules not processed for user `57c9881ba9f94c86bae34ceb4ab30924`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:01.990123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faef2c2-1a2f-464a-ad52-dabd0169882d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:01.990490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-389df5ef-6f60-4ab7-9e7a-ea4b4b83de46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:02.014653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faef2c2-1a2f-464a-ad52-dabd0169882d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.019701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926774a-1a6a-4541-b56d-f0c4dcb93758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.020357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926774a-1a6a-4541-b56d-f0c4dcb93758 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.020631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926774a-1a6a-4541-b56d-f0c4dcb93758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.020978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926774a-1a6a-4541-b56d-f0c4dcb93758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.021721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926774a-1a6a-4541-b56d-f0c4dcb93758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.023378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926774a-1a6a-4541-b56d-f0c4dcb93758 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.041519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 805/1614] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:02.045156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf824e-4916-447d-a7c2-c76bd1520514 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.046780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf824e-4916-447d-a7c2-c76bd1520514 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.047159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf824e-4916-447d-a7c2-c76bd1520514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.074178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-0faf824e-4916-447d-a7c2-c76bd1520514 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:02.075685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0faf824e-4916-447d-a7c2-c76bd1520514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:02.085540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 806/1615] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:02.086941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc74c2-722e-487f-945f-d304b608331c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.087464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc74c2-722e-487f-945f-d304b608331c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.087837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc74c2-722e-487f-945f-d304b608331c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.098297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 810/1616] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:02.100213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389df5ef-6f60-4ab7-9e7a-ea4b4b83de46 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.120602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-82bc74c2-722e-487f-945f-d304b608331c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:02.128007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-82bc74c2-722e-487f-945f-d304b608331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:02.143844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 807/1617] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:02.145227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150a5e3-51f0-4c86-a927-18010151832f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.145789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150a5e3-51f0-4c86-a927-18010151832f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.146133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150a5e3-51f0-4c86-a927-18010151832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.175098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:02.192912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389df5ef-6f60-4ab7-9e7a-ea4b4b83de46 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.197362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.197984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.198318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.198587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.213232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.214927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.215839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3150a5e3-51f0-4c86-a927-18010151832f None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:02.220165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3150a5e3-51f0-4c86-a927-18010151832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:02.229117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 811/1618] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:02.230662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.243034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851375a0-0359-4222-a8e6-f7a62c033b6b service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.248578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.249739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.250147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.250428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] PATH_INFO: `/v3/projects/9a66af4cdfa148a58816c033d86f4689/users/efe904d1cd1c42c7bc8a2923c0fab143/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.268063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a66af4cdfa148a58816c033d86f4689, user_id=efe904d1cd1c42c7bc8a2923c0fab143, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.271094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.299622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 812/1619] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:02 2022] PUT /identity/v3/projects/9a66af4cdfa148a58816c033d86f4689/users/efe904d1cd1c42c7bc8a2923c0fab143/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:02.300596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.314389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 808/1620] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 10 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:02.315492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150a5e3-51f0-4c86-a927-18010151832f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.329317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150a5e3-51f0-4c86-a927-18010151832f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.403803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef43e1-626c-41ec-85e1-27174119588f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.408488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.409929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.410350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.410727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.443899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.445326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.445651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.447452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.448226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.451581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.465466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:02.495564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.497198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.509215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 813/1621] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:02.510190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.511798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 809/1622] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:02 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:02.511943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.520027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b686c-132e-476f-ba96-f12107658717 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.582796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.583663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.584083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.584508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.615132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:02.654939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115927-f9e0-43ba-9dce-264e4650d824 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.670718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.671981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.672261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.672530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.673180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.674959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.691641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 810/1623] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:02.692804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.695428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.698566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.736441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 814/1624] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:02.745442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.805214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f670cf-ee2a-4ac0-bc89-149adb63d983 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.833367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.844337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.844666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.844976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.845711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.848216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4176e1-8c34-4e2c-a223-17261110e69e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.857089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5853c-fc65-4ea0-bdb6-e498ca4f7ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.857894 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5853c-fc65-4ea0-bdb6-e498ca4f7ae8 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.858248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5853c-fc65-4ea0-bdb6-e498ca4f7ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.858631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5853c-fc65-4ea0-bdb6-e498ca4f7ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.860217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.913838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 815/1625] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:02 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:02.915993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:02.941228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec4db2-dc89-4943-abd4-1991d6c7c4ee admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:02.953480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:02.955431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:02.956484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:02.956774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] PATH_INFO: `/v3/projects/9a66af4cdfa148a58816c033d86f4689/users/efe904d1cd1c42c7bc8a2923c0fab143/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:02.962046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a66af4cdfa148a58816c033d86f4689, user_id=efe904d1cd1c42c7bc8a2923c0fab143, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:02.969462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:02.995530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 816/1626] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:02 2022] PUT /identity/v3/projects/9a66af4cdfa148a58816c033d86f4689/users/efe904d1cd1c42c7bc8a2923c0fab143/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:03.007460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:03.027384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5853c-fc65-4ea0-bdb6-e498ca4f7ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:03.029997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5853c-fc65-4ea0-bdb6-e498ca4f7ae8 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:03.061971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 811/1627] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:03.063512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caddce2-37e3-4e17-9e7f-6a38fc25e7f8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.063904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caddce2-37e3-4e17-9e7f-6a38fc25e7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.066923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caddce2-37e3-4e17-9e7f-6a38fc25e7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.132668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2caddce2-37e3-4e17-9e7f-6a38fc25e7f8 None None] MFA Rules not processed for user `efe904d1cd1c42c7bc8a2923c0fab143`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:03.134347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2caddce2-37e3-4e17-9e7f-6a38fc25e7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:03.152000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:03.167931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956da65-913f-4adf-98cb-a3334b8d04e8 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:03.181129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:03.182011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.182468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.182917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.183713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:03.189009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:03.208007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 817/1628] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:03.215686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:03.235324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e44fc4-3fa2-4994-9961-13b834fe74c1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:03.251351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:03.254750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.255038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.255350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] PATH_INFO: `/v3/projects/aad22d1b83ef48a984a1e1a2fd00b761/users/0f708b08f6d44c4fb3e7344f36eadcad/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.260165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 812/1629] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:03.272230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caddce2-37e3-4e17-9e7f-6a38fc25e7f8 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:03.308233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad22d1b83ef48a984a1e1a2fd00b761, user_id=0f708b08f6d44c4fb3e7344f36eadcad, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:03.315004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:03.341164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 818/1630] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:03 2022] PUT /identity/v3/projects/aad22d1b83ef48a984a1e1a2fd00b761/users/0f708b08f6d44c4fb3e7344f36eadcad/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:03.345682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:03.449979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caddce2-37e3-4e17-9e7f-6a38fc25e7f8 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:03.528325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c6415-aae3-49e1-a71b-a3fccb34206e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:03.538417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cf5b7-e677-492e-9b52-1b84586935dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:03.539521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cf5b7-e677-492e-9b52-1b84586935dc admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.541245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cf5b7-e677-492e-9b52-1b84586935dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.541609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cf5b7-e677-492e-9b52-1b84586935dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.543340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5cf5b7-e677-492e-9b52-1b84586935dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:03.545084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5cf5b7-e677-492e-9b52-1b84586935dc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:03.547414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:03.561157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 819/1631] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:03.563740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70574fa6-1b56-4189-b5bd-d298f3904422 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.564695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70574fa6-1b56-4189-b5bd-d298f3904422 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.566165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70574fa6-1b56-4189-b5bd-d298f3904422 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.567900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 820/1632] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 10 12:31:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:03.568752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70574fa6-1b56-4189-b5bd-d298f3904422 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:03.571888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837bfc7-4630-41d8-9371-0d540b6801af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:03.576527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837bfc7-4630-41d8-9371-0d540b6801af service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.576920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837bfc7-4630-41d8-9371-0d540b6801af service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.577337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837bfc7-4630-41d8-9371-0d540b6801af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.587532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70574fa6-1b56-4189-b5bd-d298f3904422 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:03.604446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17fa46-8d45-4365-a87a-8bb234fc7671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:03.605114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17fa46-8d45-4365-a87a-8bb234fc7671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.606796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17fa46-8d45-4365-a87a-8bb234fc7671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.607125 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17fa46-8d45-4365-a87a-8bb234fc7671 admin admin] PATH_INFO: `/v3/projects/aad22d1b83ef48a984a1e1a2fd00b761/users/0f708b08f6d44c4fb3e7344f36eadcad/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.612727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17fa46-8d45-4365-a87a-8bb234fc7671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad22d1b83ef48a984a1e1a2fd00b761, user_id=0f708b08f6d44c4fb3e7344f36eadcad, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:03.615580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17fa46-8d45-4365-a87a-8bb234fc7671 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:03.643472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 821/1633] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:03 2022] PUT /identity/v3/projects/aad22d1b83ef48a984a1e1a2fd00b761/users/0f708b08f6d44c4fb3e7344f36eadcad/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:03.644601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129296-402e-4752-b936-70cce23f032f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.645945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129296-402e-4752-b936-70cce23f032f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.646498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129296-402e-4752-b936-70cce23f032f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.685250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-dc129296-402e-4752-b936-70cce23f032f None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:03.687837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-dc129296-402e-4752-b936-70cce23f032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:03.712986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837bfc7-4630-41d8-9371-0d540b6801af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:03.714611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837bfc7-4630-41d8-9371-0d540b6801af service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:03.731483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 813/1634] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:03.733936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a940a6-74ec-4f57-9b91-7c1d2d3af192 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.734359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a940a6-74ec-4f57-9b91-7c1d2d3af192 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.734788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a940a6-74ec-4f57-9b91-7c1d2d3af192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.765422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:03.778824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-60a940a6-74ec-4f57-9b91-7c1d2d3af192 None None] MFA Rules not processed for user `0f708b08f6d44c4fb3e7344f36eadcad`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:03.780672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-60a940a6-74ec-4f57-9b91-7c1d2d3af192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:03.830087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 822/1635] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:03.854877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129296-402e-4752-b936-70cce23f032f None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:03.890423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 814/1636] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:03.935589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a940a6-74ec-4f57-9b91-7c1d2d3af192 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:03.940409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129296-402e-4752-b936-70cce23f032f None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:03.945076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:03.946064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.947110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a940a6-74ec-4f57-9b91-7c1d2d3af192 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:03.947239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.947446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.950794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdcda0-8833-40b6-b6f8-e4e0e37e421a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:03.951413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdcda0-8833-40b6-b6f8-e4e0e37e421a service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:03.951696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdcda0-8833-40b6-b6f8-e4e0e37e421a service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:03.952006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdcda0-8833-40b6-b6f8-e4e0e37e421a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:03.961836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:03.963584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:03.980074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 823/1637] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:04.050772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdcda0-8833-40b6-b6f8-e4e0e37e421a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:04.052659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdcda0-8833-40b6-b6f8-e4e0e37e421a service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:04.066995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 815/1638] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:04.894369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:05.057966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ca51a-fac5-485e-ac33-1e4166cdd3bb service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:05.068565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:05.069410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.069695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.069974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] PATH_INFO: `/v3/users/6a3075c65d63411bb3da76276b28355c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.071633 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3075c65d63411bb3da76276b28355c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:05.086852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: WARNING py.warnings [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:31:05.086852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: warnings.warn(msg) Aug 10 12:31:05.086852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: Aug 10 12:31:05.086852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:05.159426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:05.219050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3075c65d63411bb3da76276b28355c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:05.234499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3075c65d63411bb3da76276b28355c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:05.235402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3075c65d63411bb3da76276b28355c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:05.236096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3075c65d63411bb3da76276b28355c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:05.256830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-653b1c27-b0bf-4df0-a0c4-8b6abc770f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3075c65d63411bb3da76276b28355c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:05.295551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 824/1639] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:04 2022] DELETE /identity/v3/users/6a3075c65d63411bb3da76276b28355c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:05.300742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.302182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.302870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.329244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:05.332417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f303f-b0e9-4bbb-9a61-8c4d9a6c3331 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.333177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f303f-b0e9-4bbb-9a61-8c4d9a6c3331 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.333684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f303f-b0e9-4bbb-9a61-8c4d9a6c3331 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.335848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 825/1640] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 10 12:31:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:05.336784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:05.352800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a50aa5-1f63-4229-9098-1e5e68a6596e None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.353425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a50aa5-1f63-4229-9098-1e5e68a6596e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.353982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a50aa5-1f63-4229-9098-1e5e68a6596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.392696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-91a50aa5-1f63-4229-9098-1e5e68a6596e None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:05.393848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-91a50aa5-1f63-4229-9098-1e5e68a6596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:05.464640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 816/1641] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:05.507248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:05.509177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:05.531058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 826/1642] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 10 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:05.539747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7290062-3b5d-42cd-8b79-8f46a2d71591 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.540271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7290062-3b5d-42cd-8b79-8f46a2d71591 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.540731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7290062-3b5d-42cd-8b79-8f46a2d71591 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.542792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 827/1643] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 10 12:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:31:05.555628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.556276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.557072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.600380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:05.605614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:05.613976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:05.647161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 828/1644] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:05.649192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:05.654277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:05.663883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09b09c-fb23-495e-9faa-5078d3177edf None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:05.669875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:05.671545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.672205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.673185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.688578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:05.690843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:05.705638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 817/1645] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:05.766341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a0253-d270-4429-b664-9cb7d337a878 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:05.770424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:05.770944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:05.771353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:05.771783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:05.784858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:05.786314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:05.812301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 829/1646] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:07.484837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:07.487209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:07.526525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171862de-3578-4d53-9042-a3671f959671 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:07.534480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:07.535135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:07.535460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:07.535740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] PATH_INFO: `/v3/projects/8f8ff504b0b945d9a1bcd73152eda5eb` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:07.538073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f8ff504b0b945d9a1bcd73152eda5eb)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:07.543405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: WARNING py.warnings [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 12:31:07.543405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: warnings.warn(msg) Aug 10 12:31:07.543405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: Aug 10 12:31:07.579854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:07.702006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f8ff504b0b945d9a1bcd73152eda5eb', 'request_id': 'req-2758416e-6284-49b5-a7a9-05754a7a7f17'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:07.716767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f8ff504b0b945d9a1bcd73152eda5eb', 'request_id': 'req-2758416e-6284-49b5-a7a9-05754a7a7f17'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:07.731746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f8ff504b0b945d9a1bcd73152eda5eb', 'request_id': 'req-2758416e-6284-49b5-a7a9-05754a7a7f17'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:07.777848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] The token cache is being invalidate because project 8f8ff504b0b945d9a1bcd73152eda5eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:07.778147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:07.783275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:07.784941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 818/1647] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:07 2022] DELETE /identity/v3/projects/8f8ff504b0b945d9a1bcd73152eda5eb => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:07.788965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:07.810891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94a79b-bab3-4cda-bd2a-89a9520e1454 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:07.873286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:08.063758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:08.064534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:08.064937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:08.065348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] PATH_INFO: `/v3/users/efe904d1cd1c42c7bc8a2923c0fab143` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:08.066698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe904d1cd1c42c7bc8a2923c0fab143)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:08.069382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:08.212874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe904d1cd1c42c7bc8a2923c0fab143'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:08.238767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe904d1cd1c42c7bc8a2923c0fab143'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:08.241337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe904d1cd1c42c7bc8a2923c0fab143'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:08.242105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe904d1cd1c42c7bc8a2923c0fab143'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:08.265702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-75d2b2fe-069a-4d8d-90f4-af4673c3bfd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe904d1cd1c42c7bc8a2923c0fab143'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:08.323531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 830/1648] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:07 2022] DELETE /identity/v3/users/efe904d1cd1c42c7bc8a2923c0fab143 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:10.023909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:10.025404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:10.044964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758416e-6284-49b5-a7a9-05754a7a7f17 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:10.234248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:10.235452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:10.235868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:10.236281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] PATH_INFO: `/v3/projects/9a66af4cdfa148a58816c033d86f4689` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:10.237632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a66af4cdfa148a58816c033d86f4689)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:10.239736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:10.355656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a66af4cdfa148a58816c033d86f4689', 'request_id': 'req-e073d62f-bffd-4077-a99d-7f752123de54'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:10.363670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a66af4cdfa148a58816c033d86f4689', 'request_id': 'req-e073d62f-bffd-4077-a99d-7f752123de54'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:10.368535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:10.376976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a66af4cdfa148a58816c033d86f4689', 'request_id': 'req-e073d62f-bffd-4077-a99d-7f752123de54'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:10.411218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] The token cache is being invalidate because project 9a66af4cdfa148a58816c033d86f4689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:10.411518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:10.412999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e073d62f-bffd-4077-a99d-7f752123de54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:10.420472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 819/1649] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:10 2022] DELETE /identity/v3/projects/9a66af4cdfa148a58816c033d86f4689 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:10.425724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3f5a5-f5c5-49ab-999f-a70ff40d42df None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:10.426551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3f5a5-f5c5-49ab-999f-a70ff40d42df None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:10.426961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3f5a5-f5c5-49ab-999f-a70ff40d42df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:10.470512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-00f3f5a5-f5c5-49ab-999f-a70ff40d42df None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:10.471756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-00f3f5a5-f5c5-49ab-999f-a70ff40d42df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:10.643186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 831/1650] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:10.645714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b503731-db47-4ba5-affc-0f0008145fac None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:10.646000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b503731-db47-4ba5-affc-0f0008145fac None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:10.648555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b503731-db47-4ba5-affc-0f0008145fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:10.694540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8b503731-db47-4ba5-affc-0f0008145fac None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:10.700243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8b503731-db47-4ba5-affc-0f0008145fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:10.719596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 820/1651] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:10.723562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3f5a5-f5c5-49ab-999f-a70ff40d42df None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:10.741467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3f5a5-f5c5-49ab-999f-a70ff40d42df None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:10.752116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:10.752753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:10.753457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:10.753457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:10.754066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:10.754364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:10.755995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:10.788783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 832/1652] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:10.793715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b503731-db47-4ba5-affc-0f0008145fac None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:10.812623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b503731-db47-4ba5-affc-0f0008145fac None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:10.822041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:10.822683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:10.822957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:10.823225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:10.823877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:10.827067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:10.894055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 821/1653] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:10 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:10.898176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:10.913077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0309f-d434-4cdc-8793-d6cd201114fb admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:10.998736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:11.076392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:11.077071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:11.077372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:11.077832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:11.078492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:11.080474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:11.142678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 833/1654] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:10 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:11.143969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:11.164774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aa08f-34ba-46d8-bfdb-8dc37baa4129 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:11.179164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:11.179979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:11.180391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:11.180791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:11.181767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:11.183533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:11.222323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 822/1655] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:11.227721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:11.240910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0fd65-29ed-4746-8f69-0d75ae8013db admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:11.255699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffaaec-caec-44fe-b72d-7cb3fea30d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:11.256334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffaaec-caec-44fe-b72d-7cb3fea30d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:11.256614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffaaec-caec-44fe-b72d-7cb3fea30d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:11.256882 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffaaec-caec-44fe-b72d-7cb3fea30d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:11.257548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffaaec-caec-44fe-b72d-7cb3fea30d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:11.259176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffaaec-caec-44fe-b72d-7cb3fea30d20 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:11.284889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 834/1656] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:11.289782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:11.306556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e64bd-f5eb-4b8e-b2bf-be6588036565 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:11.320814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:11.321552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:11.321832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:11.322102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] PATH_INFO: `/v3/projects/160cecf9ac214a48ba431776533c86df/users/3b7f061737fe4a49858cf7c80c862177/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:11.355306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] RBAC: Authorizing `identity:create_grant(project_id=160cecf9ac214a48ba431776533c86df, user_id=3b7f061737fe4a49858cf7c80c862177, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:11.358963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:11.408459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 823/1657] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:11 2022] PUT /identity/v3/projects/160cecf9ac214a48ba431776533c86df/users/3b7f061737fe4a49858cf7c80c862177/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:11.409982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cbd17-0b40-430f-8a02-ec685f4f454a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:11.410282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cbd17-0b40-430f-8a02-ec685f4f454a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:11.410551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cbd17-0b40-430f-8a02-ec685f4f454a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:11.469052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fa1cbd17-0b40-430f-8a02-ec685f4f454a None None] MFA Rules not processed for user `3b7f061737fe4a49858cf7c80c862177`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:11.470307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fa1cbd17-0b40-430f-8a02-ec685f4f454a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:11.691002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 835/1658] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:11.692437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:11.816024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98d11c-583c-419c-a442-fba18c94afde admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:11.833145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:11.834071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:11.834482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:11.834834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:11.835540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:11.837598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:11.900351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 824/1659] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:11 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:11.908278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cbd17-0b40-430f-8a02-ec685f4f454a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:11.970884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cbd17-0b40-430f-8a02-ec685f4f454a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:12.300132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:12.300490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:12.300776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:12.301047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:12.302071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:12.304041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:12.341165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 836/1660] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:11 2022] POST /identity/v3/users => generated 292 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:12.341676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:12.355757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8917bc-a5ee-44ee-8b93-8b0a269b3f20 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:12.363636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:12.364634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:12.365112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:12.365640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:12.366491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:12.370508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:12.393849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:12.405328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 825/1661] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:12.408392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cc02f-2727-4421-a185-1aaf14526060 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:12.414383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:12.415352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:12.416042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:12.416530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] PATH_INFO: `/v3/projects/371d2c54095f4a4b884ff849c1c70fbd/users/3a3cdebffe274cb19569d9bccb949ce1/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:12.442149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=371d2c54095f4a4b884ff849c1c70fbd, user_id=3a3cdebffe274cb19569d9bccb949ce1, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:12.445688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:12.478336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:12.489509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 837/1662] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:12 2022] PUT /identity/v3/projects/371d2c54095f4a4b884ff849c1c70fbd/users/3a3cdebffe274cb19569d9bccb949ce1/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:12.493576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be807a40-42b9-4ed0-a83c-3db5f7c21c21 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:12.571432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7115a-6ac1-4c64-aca8-f47eb7ac4db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:12.572069 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7115a-6ac1-4c64-aca8-f47eb7ac4db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:12.579758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7115a-6ac1-4c64-aca8-f47eb7ac4db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:12.580081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7115a-6ac1-4c64-aca8-f47eb7ac4db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:12.580806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7115a-6ac1-4c64-aca8-f47eb7ac4db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:12.582512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7115a-6ac1-4c64-aca8-f47eb7ac4db4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:12.600444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:12.622178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 826/1663] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:12.624831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:12.634141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:12.644271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9fb19-1942-471f-8f78-70baeb89e31e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:12.651913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:12.653575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:12.654374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:12.655660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] PATH_INFO: `/v3/projects/371d2c54095f4a4b884ff849c1c70fbd/users/3a3cdebffe274cb19569d9bccb949ce1/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:12.659877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=371d2c54095f4a4b884ff849c1c70fbd, user_id=3a3cdebffe274cb19569d9bccb949ce1, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:12.666237 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:12.729060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 838/1664] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:12 2022] PUT /identity/v3/projects/371d2c54095f4a4b884ff849c1c70fbd/users/3a3cdebffe274cb19569d9bccb949ce1/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:12.743430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abc13a-204a-4cce-a7c4-578ddce80b46 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:12.743667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abc13a-204a-4cce-a7c4-578ddce80b46 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:12.743947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abc13a-204a-4cce-a7c4-578ddce80b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:12.797116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-59abc13a-204a-4cce-a7c4-578ddce80b46 None None] MFA Rules not processed for user `3a3cdebffe274cb19569d9bccb949ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:12.799871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-59abc13a-204a-4cce-a7c4-578ddce80b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:12.968828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 827/1665] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:12.981473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:12.984456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:13.103484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad92c9-ef25-45d0-b24f-8af91fdc2a12 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:13.107242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:13.107883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:13.108708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:13.108990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:13.124429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:13.125944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:13.141499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 839/1666] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:13.857889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633952-97d4-4ece-9cab-f43c833f0eb4 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:13.859470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633952-97d4-4ece-9cab-f43c833f0eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:13.860680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633952-97d4-4ece-9cab-f43c833f0eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:13.987626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-00633952-97d4-4ece-9cab-f43c833f0eb4 None None] MFA Rules not processed for user `94473332fd86422a9f1a1eed466f5a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:13.992670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-00633952-97d4-4ece-9cab-f43c833f0eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:14.274298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:14.311722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 828/1667] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:14.327935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:14.328430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:14.328520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:14.399130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] MFA Rules not processed for user `94473332fd86422a9f1a1eed466f5a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:14.404994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:14.422933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 829/1668] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:14.423942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:14.430343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:14.467972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b12837-70ad-4a50-adcd-ff80f83b7901 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:14.493893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:14.496864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:14.497459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:14.500031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:14.505971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd1d5c-5fa5-4b1c-a1a0-8ca6e3b8f707 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:14.506875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:14.516268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:14.517073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:14.517542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:14.518874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] PATH_INFO: `/v3/users/3b7f061737fe4a49858cf7c80c862177` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:14.524569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7f061737fe4a49858cf7c80c862177)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:14.538615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:14.714655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7f061737fe4a49858cf7c80c862177'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:14.737815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:14.739476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:14.742840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7f061737fe4a49858cf7c80c862177'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:14.743496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7f061737fe4a49858cf7c80c862177'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:14.745716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7f061737fe4a49858cf7c80c862177'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:14.751693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:14.761067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-12046be6-b717-44e7-b01a-62ade0e0c5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7f061737fe4a49858cf7c80c862177'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:14.767665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 830/1669] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:14.768497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:14.772935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:14.824002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 840/1670] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:14 2022] DELETE /identity/v3/users/3b7f061737fe4a49858cf7c80c862177 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:14.829003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:14.829262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:14.829563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:14.902236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:14.905232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:14.949538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:15.064415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6183668-b193-4a25-9502-2bf45efef852 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:15.156889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 841/1671] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:15.185292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:15.186850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:15.326301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ce3ac-66e4-4b91-b3f8-ee5ab2041f1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:15.327454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce3ac-66e4-4b91-b3f8-ee5ab2041f1f service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:15.327799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce3ac-66e4-4b91-b3f8-ee5ab2041f1f service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:15.328135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce3ac-66e4-4b91-b3f8-ee5ab2041f1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:15.331648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e28ce3ac-66e4-4b91-b3f8-ee5ab2041f1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:15.351524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068496f-c4c4-47f7-b116-fc7c010e19f5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:15.364003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8522a0-f6da-4629-9570-e7dd1a9d6e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:15.364655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8522a0-f6da-4629-9570-e7dd1a9d6e0d service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:15.365010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8522a0-f6da-4629-9570-e7dd1a9d6e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:15.367592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8522a0-f6da-4629-9570-e7dd1a9d6e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:15.390311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8522a0-f6da-4629-9570-e7dd1a9d6e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:15.395541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8522a0-f6da-4629-9570-e7dd1a9d6e0d service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:15.436223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 842/1672] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:15.509225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ce3ac-66e4-4b91-b3f8-ee5ab2041f1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:15.509225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ce3ac-66e4-4b91-b3f8-ee5ab2041f1f service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:15.555797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 831/1673] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 788 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:15.581422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:15.583388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:15.584219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:15.677583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] MFA Rules not processed for user `a78bd1aed509474b965592c478cb1c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:15.687377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:15.909413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 843/1674] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:15.926852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:15.927404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:15.927691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:15.978616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] MFA Rules not processed for user `9b8e4aad09814f8299a334639e34d948`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:15.981250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:16.010657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:16.087989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 832/1675] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:16.117187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:16.118836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:16.145249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:16.147076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:16.148017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900a345-deb8-4b56-a83f-5dcd49c978c4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:16.154691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:16.158598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:16.158941 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:16.159222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] PATH_INFO: `/v3/users/9ea99f7a48134f2b8a36f5d67c22ad51` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:16.161749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea99f7a48134f2b8a36f5d67c22ad51)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:16.163788 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:16.177367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45804-0e79-4de8-a753-4fd897fdf64a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:16.206009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254429f-474d-410a-9774-a95f9ea55d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:16.206652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254429f-474d-410a-9774-a95f9ea55d35 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:16.206932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254429f-474d-410a-9774-a95f9ea55d35 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:16.207207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254429f-474d-410a-9774-a95f9ea55d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:16.232776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254429f-474d-410a-9774-a95f9ea55d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:16.238332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254429f-474d-410a-9774-a95f9ea55d35 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:16.265687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 833/1676] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:16.299266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:16.331448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea99f7a48134f2b8a36f5d67c22ad51'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:16.360816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea99f7a48134f2b8a36f5d67c22ad51'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:16.361591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea99f7a48134f2b8a36f5d67c22ad51'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:16.362130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea99f7a48134f2b8a36f5d67c22ad51'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:16.375055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea99f7a48134f2b8a36f5d67c22ad51'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:16.419097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 844/1677] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:16 2022] DELETE /identity/v3/users/9ea99f7a48134f2b8a36f5d67c22ad51 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:16.423948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:16.424413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:16.424765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:16.480419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:16.490261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:16.698467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 834/1678] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:16.735889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:16.738480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:16.845751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b44241-5c2d-444d-a747-e3aef4d904fa admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:16.850302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25a5bb-7ff6-467b-8aa1-6bfd6b280aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:16.850938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25a5bb-7ff6-467b-8aa1-6bfd6b280aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:16.851262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25a5bb-7ff6-467b-8aa1-6bfd6b280aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:16.851537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25a5bb-7ff6-467b-8aa1-6bfd6b280aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:16.892366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25a5bb-7ff6-467b-8aa1-6bfd6b280aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:16.894432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25a5bb-7ff6-467b-8aa1-6bfd6b280aa5 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:16.908981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:16.910636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:16.915326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 845/1679] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:17.069568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712eb65-1218-4715-a271-67c72a7c4be0 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:17.069884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712eb65-1218-4715-a271-67c72a7c4be0 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:17.070187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712eb65-1218-4715-a271-67c72a7c4be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:17.108674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de889cd-004c-44ce-81c7-2e80c417de3c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:17.122823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f36c9-2a34-4bfc-b049-788f292683f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:17.124687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f36c9-2a34-4bfc-b049-788f292683f0 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:17.125746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f36c9-2a34-4bfc-b049-788f292683f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:17.126277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f36c9-2a34-4bfc-b049-788f292683f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:17.130344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-613f36c9-2a34-4bfc-b049-788f292683f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:17.140301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-1712eb65-1218-4715-a271-67c72a7c4be0 None None] MFA Rules not processed for user `2aa6d6d56f9f4618ae3883c9beb2e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:17.141791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1712eb65-1218-4715-a271-67c72a7c4be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:17.287145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:17.314128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f36c9-2a34-4bfc-b049-788f292683f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:17.318134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f36c9-2a34-4bfc-b049-788f292683f0 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:17.348198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 846/1680] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:17.349600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32eba9-0223-4b6a-8072-b2a11aa1ad33 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:17.352632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32eba9-0223-4b6a-8072-b2a11aa1ad33 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:17.353313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32eba9-0223-4b6a-8072-b2a11aa1ad33 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:17.357438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 847/1681] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 10 12:31:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:17.361872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 835/1682] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:17.362183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:17.362616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:17.362995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:17.365667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:17.365973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:17.367655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:17.411478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] MFA Rules not processed for user `2aa6d6d56f9f4618ae3883c9beb2e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:17.415773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:17.446951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 848/1683] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:17.505031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:17.507881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:17.550689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] MFA Rules not processed for user `8c4588df9de242e8819de0476f75959c`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:17.552526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d54c-b094-4d82-b1f6-fb3668355d7d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:17.554093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:17.574225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:17.574930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:17.575153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:17.575433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] PATH_INFO: `/v3/projects/160cecf9ac214a48ba431776533c86df` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:17.578004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=160cecf9ac214a48ba431776533c86df)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:17.581511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:17.581958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 836/1684] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:17.650204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:17.652133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:17.670104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466c19a-4650-4e88-802c-592998aafae7 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:17.697588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:17.698226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:17.698887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:17.700948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:17.722373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:17.736224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160cecf9ac214a48ba431776533c86df', 'request_id': 'req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:17.739413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:17.743513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:17.762808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160cecf9ac214a48ba431776533c86df', 'request_id': 'req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:17.767018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 837/1685] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:17.768927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:17.783420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160cecf9ac214a48ba431776533c86df', 'request_id': 'req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:17.803652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] The token cache is being invalidate because project 160cecf9ac214a48ba431776533c86df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:17.803968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:17.804622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:17.806193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 849/1686] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:17 2022] DELETE /identity/v3/projects/160cecf9ac214a48ba431776533c86df => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:17.810531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac00e9f-b7d9-4faf-98ec-7bc949669d06 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:17.810806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:17.820117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:17.995219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbdfa6-7f1b-4250-9cdb-c33a6ecfc1d2 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:18.002007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:18.003011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:18.003489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:18.003960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] PATH_INFO: `/v3/users/3a3cdebffe274cb19569d9bccb949ce1` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:18.005513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3cdebffe274cb19569d9bccb949ce1)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:18.007397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a33dfb-13e0-4e8f-9c5c-e9d8ded0d40e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:18.008082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33dfb-13e0-4e8f-9c5c-e9d8ded0d40e service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:18.008288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:18.008382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33dfb-13e0-4e8f-9c5c-e9d8ded0d40e service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:18.008619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33dfb-13e0-4e8f-9c5c-e9d8ded0d40e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:18.012661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-45a33dfb-13e0-4e8f-9c5c-e9d8ded0d40e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:18.116273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3cdebffe274cb19569d9bccb949ce1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.142974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3cdebffe274cb19569d9bccb949ce1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.143641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3cdebffe274cb19569d9bccb949ce1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.144341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3cdebffe274cb19569d9bccb949ce1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.158195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3cdebffe274cb19569d9bccb949ce1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.196356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a33dfb-13e0-4e8f-9c5c-e9d8ded0d40e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:18.200416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a33dfb-13e0-4e8f-9c5c-e9d8ded0d40e service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:18.208053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 850/1687] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:17 2022] DELETE /identity/v3/users/3a3cdebffe274cb19569d9bccb949ce1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:18.208053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:18.208590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:18.224480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 838/1688] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:18.333599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:18.388441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8b964-fd90-4ccd-869d-fc5823f1c03c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:18.410706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:18.411387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:18.411671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:18.411970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] PATH_INFO: `/v3/projects/a99cd651b5c8474fb336b307cbba38cd` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:18.418253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99cd651b5c8474fb336b307cbba38cd)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:18.421964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:18.567182 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99cd651b5c8474fb336b307cbba38cd', 'request_id': 'req-5f3cab33-a79c-4011-be89-3000adaa645b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.590948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99cd651b5c8474fb336b307cbba38cd', 'request_id': 'req-5f3cab33-a79c-4011-be89-3000adaa645b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.620058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99cd651b5c8474fb336b307cbba38cd', 'request_id': 'req-5f3cab33-a79c-4011-be89-3000adaa645b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.639336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] The token cache is being invalidate because project a99cd651b5c8474fb336b307cbba38cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:18.639720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.641596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f3cab33-a79c-4011-be89-3000adaa645b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:18.646924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 851/1689] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:18 2022] DELETE /identity/v3/projects/a99cd651b5c8474fb336b307cbba38cd => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:18.651991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609dc28-4e80-4e33-8d80-b2dd64624ade None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:18.652271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609dc28-4e80-4e33-8d80-b2dd64624ade None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:18.652552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609dc28-4e80-4e33-8d80-b2dd64624ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:18.692529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5609dc28-4e80-4e33-8d80-b2dd64624ade None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:18.693841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5609dc28-4e80-4e33-8d80-b2dd64624ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:18.812224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:18.849644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 839/1690] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:18.850808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7d35b-bdd6-4cd9-b7c0-85ccaa394a86 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:18.851143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7d35b-bdd6-4cd9-b7c0-85ccaa394a86 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:18.851421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7d35b-bdd6-4cd9-b7c0-85ccaa394a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:18.898555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d5f7d35b-bdd6-4cd9-b7c0-85ccaa394a86 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:18.901204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d5f7d35b-bdd6-4cd9-b7c0-85ccaa394a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:18.926653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 852/1691] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:18.928197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609dc28-4e80-4e33-8d80-b2dd64624ade None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:18.943991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609dc28-4e80-4e33-8d80-b2dd64624ade None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:18.953508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:18.954672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:18.955014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:18.955336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:18.956443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:18.956785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:18.981573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.008679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7d35b-bdd6-4cd9-b7c0-85ccaa394a86 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.009183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 840/1692] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:19.025667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7d35b-bdd6-4cd9-b7c0-85ccaa394a86 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:19.036355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca482c-49bd-4497-b901-e4dfed2c6c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:19.037366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca482c-49bd-4497-b901-e4dfed2c6c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.037765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca482c-49bd-4497-b901-e4dfed2c6c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.045650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca482c-49bd-4497-b901-e4dfed2c6c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.046593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca482c-49bd-4497-b901-e4dfed2c6c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:19.048799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca482c-49bd-4497-b901-e4dfed2c6c02 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.118486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 853/1693] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:19 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:19.119775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.137523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2020a7-1cbc-47e6-9023-cc89d387494d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:19.197006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b39e87-94e9-48bb-ac48-8eda3ac66061 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.197308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b39e87-94e9-48bb-ac48-8eda3ac66061 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.197596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b39e87-94e9-48bb-ac48-8eda3ac66061 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.199380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 854/1694] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 10 12:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:31:19.205576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043a24f-a133-4ae8-ae02-accb7c96ec94 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.205945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043a24f-a133-4ae8-ae02-accb7c96ec94 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.206330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043a24f-a133-4ae8-ae02-accb7c96ec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.257636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a043a24f-a133-4ae8-ae02-accb7c96ec94 None None] MFA Rules not processed for user `8c4588df9de242e8819de0476f75959c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:19.257735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff5662-845a-4909-8b79-2b6585d53eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:19.257735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5662-845a-4909-8b79-2b6585d53eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.257735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5662-845a-4909-8b79-2b6585d53eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.257735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5662-845a-4909-8b79-2b6585d53eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.258366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff5662-845a-4909-8b79-2b6585d53eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:19.258451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a043a24f-a133-4ae8-ae02-accb7c96ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:19.259104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff5662-845a-4909-8b79-2b6585d53eb3 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.314622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 841/1695] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:19 2022] POST /identity/v3/users => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:19.319141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682725-89a4-4530-9d58-bf29bae049f9 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.319598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682725-89a4-4530-9d58-bf29bae049f9 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.321077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682725-89a4-4530-9d58-bf29bae049f9 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.324588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 842/1696] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 10 12:31:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:19.326540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682725-89a4-4530-9d58-bf29bae049f9 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.357702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682725-89a4-4530-9d58-bf29bae049f9 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:19.364766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a6ca5-0b9c-4b2e-a604-9e1c328c5327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:19.365667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a6ca5-0b9c-4b2e-a604-9e1c328c5327 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.365976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a6ca5-0b9c-4b2e-a604-9e1c328c5327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.366317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a6ca5-0b9c-4b2e-a604-9e1c328c5327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.367151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a6ca5-0b9c-4b2e-a604-9e1c328c5327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:19.374299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a6ca5-0b9c-4b2e-a604-9e1c328c5327 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.391117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 843/1697] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:19.392614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f309af-da7f-4598-ac91-669a71b9159c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.393030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f309af-da7f-4598-ac91-669a71b9159c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.393470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f309af-da7f-4598-ac91-669a71b9159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.415257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-85f309af-da7f-4598-ac91-669a71b9159c None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:19.416580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-85f309af-da7f-4598-ac91-669a71b9159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:19.446661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 855/1698] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:19.449323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043a24f-a133-4ae8-ae02-accb7c96ec94 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.487502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043a24f-a133-4ae8-ae02-accb7c96ec94 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:19.493978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:19.495420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.495914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.497118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.498426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:19.502993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.528686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 856/1699] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:19.533130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.535364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:19.543461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 844/1700] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 10 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:19.544322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f309af-da7f-4598-ac91-669a71b9159c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.553815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d528e5-05e3-422a-83a5-d36e48ef064e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:19.558258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:19.559349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.559903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.560629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.571749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:19.583756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.603359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f309af-da7f-4598-ac91-669a71b9159c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:19.612765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 857/1701] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:19.613406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.617497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4504bc-fab4-4226-8280-5a7ac8700d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:19.618192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4504bc-fab4-4226-8280-5a7ac8700d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.618531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4504bc-fab4-4226-8280-5a7ac8700d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.621098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4504bc-fab4-4226-8280-5a7ac8700d3d admin admin] PATH_INFO: `/v3/projects/e47542b2b1d64168b2846b18b22eadbf/users/a22766def69f46df92a8bf657f167402/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.642707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2edc5-f7b1-432a-a8fd-c912cfa5c6c1 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:19.651401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4504bc-fab4-4226-8280-5a7ac8700d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47542b2b1d64168b2846b18b22eadbf, user_id=a22766def69f46df92a8bf657f167402, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:19.654159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4504bc-fab4-4226-8280-5a7ac8700d3d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.657237 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:19.657885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.658293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.658974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.665064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:19.693025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 845/1702] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:19 2022] PUT /identity/v3/projects/e47542b2b1d64168b2846b18b22eadbf/users/a22766def69f46df92a8bf657f167402/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:19.694550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:19.694836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:19.695109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:19.700930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:19.702579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:19.727990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 858/1703] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:19.734610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:19.735904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:19.829901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:19.937132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 846/1704] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:19.958236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:19.974503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d57aa1-921b-40dc-b5f3-f34d30aebac4 service placement] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:20.010852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:20.090161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e713f2-b7c7-4734-87c3-2543cc34e812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:20.102172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e713f2-b7c7-4734-87c3-2543cc34e812 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:20.103293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e713f2-b7c7-4734-87c3-2543cc34e812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:20.103728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e713f2-b7c7-4734-87c3-2543cc34e812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:20.133956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e713f2-b7c7-4734-87c3-2543cc34e812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:20.136207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e713f2-b7c7-4734-87c3-2543cc34e812 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:20.182974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 859/1705] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:20.776047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:20.778907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:20.800375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9a734-80d9-477b-a4cc-53ec80ffe57e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:20.814978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:20.815962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:20.816314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:20.816675 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] PATH_INFO: `/v3/projects/371d2c54095f4a4b884ff849c1c70fbd` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:20.821592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=371d2c54095f4a4b884ff849c1c70fbd)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:20.823940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:20.933017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371d2c54095f4a4b884ff849c1c70fbd', 'request_id': 'req-3aa750a7-5fe1-4069-8858-469b9f561cf1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:20.946740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371d2c54095f4a4b884ff849c1c70fbd', 'request_id': 'req-3aa750a7-5fe1-4069-8858-469b9f561cf1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:20.967839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371d2c54095f4a4b884ff849c1c70fbd', 'request_id': 'req-3aa750a7-5fe1-4069-8858-469b9f561cf1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:20.991308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] The token cache is being invalidate because project 371d2c54095f4a4b884ff849c1c70fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:20.991756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:20.994215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3aa750a7-5fe1-4069-8858-469b9f561cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:20.999598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 847/1706] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:20 2022] DELETE /identity/v3/projects/371d2c54095f4a4b884ff849c1c70fbd => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:21.004861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac366d0-9ed3-4983-be5a-7d1e135de268 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:21.005475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac366d0-9ed3-4983-be5a-7d1e135de268 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:21.005475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac366d0-9ed3-4983-be5a-7d1e135de268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:21.052316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-bac366d0-9ed3-4983-be5a-7d1e135de268 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:21.053680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bac366d0-9ed3-4983-be5a-7d1e135de268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:21.190924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 860/1707] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:21.202147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03047e7-142a-470b-8b99-d3a9437679ba None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:21.205718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03047e7-142a-470b-8b99-d3a9437679ba None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:21.206236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03047e7-142a-470b-8b99-d3a9437679ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:21.286794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44e14-f093-49a2-baff-86a1949042c2 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:21.286996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44e14-f093-49a2-baff-86a1949042c2 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:21.287278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44e14-f093-49a2-baff-86a1949042c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:21.364753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a03047e7-142a-470b-8b99-d3a9437679ba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:21.366197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a03047e7-142a-470b-8b99-d3a9437679ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:21.398588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-3ca44e14-f093-49a2-baff-86a1949042c2 None None] MFA Rules not processed for user `2aa6d6d56f9f4618ae3883c9beb2e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:21.411751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3ca44e14-f093-49a2-baff-86a1949042c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:21.416875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 848/1708] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:21.421616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03047e7-142a-470b-8b99-d3a9437679ba None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:21.488851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03047e7-142a-470b-8b99-d3a9437679ba None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:21.530417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:21.536638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:21.537568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:21.555509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:21.559364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:21.564391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:21.571740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:21.643403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 861/1709] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:21.667687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44e14-f093-49a2-baff-86a1949042c2 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:21.669021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 849/1710] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:21.691041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44e14-f093-49a2-baff-86a1949042c2 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:21.704889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:21.705545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:21.705827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:21.706095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:21.706727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:21.708664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:21.833159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 862/1711] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:21 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:21.835446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:21.856335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b5411-e9fc-4e0a-a00d-d1177588bfc7 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:22.034218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:22.035111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:22.035542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:22.035962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:22.036752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:22.038889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:22.121737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 850/1712] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:21 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:22.126288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:22.174866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20eda30-62bd-40ed-b0c9-a91a345b5118 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:22.200282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:22.204911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:22.205192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:22.205497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:22.206173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:22.227667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:22.278400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 863/1713] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:22.283235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:22.297555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291e28c-fb9f-4f39-9ebc-1a8d9a59f0bb admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:22.322531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838a016-3b35-41fa-89a8-97ffe0cdc464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:22.323345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838a016-3b35-41fa-89a8-97ffe0cdc464 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:22.323771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838a016-3b35-41fa-89a8-97ffe0cdc464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:22.325475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838a016-3b35-41fa-89a8-97ffe0cdc464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:22.326141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838a016-3b35-41fa-89a8-97ffe0cdc464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:22.338404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838a016-3b35-41fa-89a8-97ffe0cdc464 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:22.377083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:22.378524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 851/1714] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:22.392385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed5b54-4ec9-42dd-ade0-7f592d1a1ea1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:22.410334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:22.412404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:22.416252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:22.416649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] PATH_INFO: `/v3/projects/c6cb718b290042b78821ff93f238d2c0/users/b8d777e32b98472fb09b55ef10a997e4/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:22.478262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cb718b290042b78821ff93f238d2c0, user_id=b8d777e32b98472fb09b55ef10a997e4, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:22.481857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:22.533321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 864/1715] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:22 2022] PUT /identity/v3/projects/c6cb718b290042b78821ff93f238d2c0/users/b8d777e32b98472fb09b55ef10a997e4/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:22.541307 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d59f03-6920-4740-aecd-e1d057f5908c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:22.541827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d59f03-6920-4740-aecd-e1d057f5908c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:22.542255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d59f03-6920-4740-aecd-e1d057f5908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:22.621034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:22.630444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d2d59f03-6920-4740-aecd-e1d057f5908c None None] MFA Rules not processed for user `b8d777e32b98472fb09b55ef10a997e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:22.635620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d2d59f03-6920-4740-aecd-e1d057f5908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:22.781331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 852/1716] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:22.785775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:22.805526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:22.957947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705558c-4ab8-4903-8b9a-cf3cb09a78d9 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:22.964417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:22.965577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:22.966089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:22.966584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:22.967550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:22.970219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:23.029778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 865/1717] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:22 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:23.034090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d59f03-6920-4740-aecd-e1d057f5908c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:23.055627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d59f03-6920-4740-aecd-e1d057f5908c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:23.153187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51be52-1dea-4ace-b2dd-dcf50c78f896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:23.154229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51be52-1dea-4ace-b2dd-dcf50c78f896 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:23.155591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51be52-1dea-4ace-b2dd-dcf50c78f896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:23.155887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51be52-1dea-4ace-b2dd-dcf50c78f896 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:23.157353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51be52-1dea-4ace-b2dd-dcf50c78f896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:23.163196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51be52-1dea-4ace-b2dd-dcf50c78f896 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:23.223603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 853/1718] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:23 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:23.227386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:23.254251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60582599-ced1-489d-8d11-6a69963bbf07 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:23.260217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:23.261324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:23.261857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:23.282860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb664e-7406-45f0-b2e0-03810bf552dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:23.284715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb664e-7406-45f0-b2e0-03810bf552dd admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:23.285401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb664e-7406-45f0-b2e0-03810bf552dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:23.285826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb664e-7406-45f0-b2e0-03810bf552dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:23.288504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb664e-7406-45f0-b2e0-03810bf552dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:23.290385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb664e-7406-45f0-b2e0-03810bf552dd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:23.304763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] MFA Rules not processed for user `a22766def69f46df92a8bf657f167402`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:23.309660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:23.319016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 866/1719] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:23.320703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f432-afc2-4c6d-a5bd-646787a11a43 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:23.321146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f432-afc2-4c6d-a5bd-646787a11a43 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:23.321709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f432-afc2-4c6d-a5bd-646787a11a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:23.366955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ff85f432-afc2-4c6d-a5bd-646787a11a43 None None] MFA Rules not processed for user `a78bd1aed509474b965592c478cb1c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:23.368820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ff85f432-afc2-4c6d-a5bd-646787a11a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:23.413679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 854/1720] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:23.420063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:23.424029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:23.454786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9370b4-bfb9-4a9d-aef6-052c627860f0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:23.465531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:23.469329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 867/1721] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:23.469453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:23.469971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:23.470332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f432-afc2-4c6d-a5bd-646787a11a43 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:23.470458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] PATH_INFO: `/v3/users/c215a6f4372240cb914577c37e3aee30` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:23.472184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c215a6f4372240cb914577c37e3aee30)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:23.474900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:23.506694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f432-afc2-4c6d-a5bd-646787a11a43 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:23.523389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:23.536103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960075b2-e038-4029-92d0-182827811618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:23.539629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960075b2-e038-4029-92d0-182827811618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:23.539978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960075b2-e038-4029-92d0-182827811618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:23.540270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960075b2-e038-4029-92d0-182827811618 admin admin] PATH_INFO: `/v3/projects/c1f924b486d44595a251553e9542fe8a/users/9978583e383b46f79c6ad21badaa8514/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:23.601714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960075b2-e038-4029-92d0-182827811618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f924b486d44595a251553e9542fe8a, user_id=9978583e383b46f79c6ad21badaa8514, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:23.604566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960075b2-e038-4029-92d0-182827811618 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:23.632790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c215a6f4372240cb914577c37e3aee30'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:23.650420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c215a6f4372240cb914577c37e3aee30'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:23.661307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 868/1722] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:23 2022] PUT /identity/v3/projects/c1f924b486d44595a251553e9542fe8a/users/9978583e383b46f79c6ad21badaa8514/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:23.663738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960075b2-e038-4029-92d0-182827811618 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:23.667721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-960075b2-e038-4029-92d0-182827811618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:23.677473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c215a6f4372240cb914577c37e3aee30'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:23.678457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c215a6f4372240cb914577c37e3aee30'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:23.680126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c215a6f4372240cb914577c37e3aee30'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:23.735780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 855/1723] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:23 2022] DELETE /identity/v3/users/c215a6f4372240cb914577c37e3aee30 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:23.739901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:23.741817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:23.821038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960075b2-e038-4029-92d0-182827811618 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:23.843072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:23.950766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe466-ce2a-4f5f-853e-be0b26a780e4 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:23.964405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f4966-aaa0-41d9-8513-7fc896ec57c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:23.965008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4966-aaa0-41d9-8513-7fc896ec57c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:23.965366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4966-aaa0-41d9-8513-7fc896ec57c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:23.965647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4966-aaa0-41d9-8513-7fc896ec57c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:23.966411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f4966-aaa0-41d9-8513-7fc896ec57c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:23.969119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f4966-aaa0-41d9-8513-7fc896ec57c4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:24.006977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 856/1724] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:24.010397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355a0b-7062-4bef-8df1-22850dbea8c6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.010699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355a0b-7062-4bef-8df1-22850dbea8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.010983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355a0b-7062-4bef-8df1-22850dbea8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.016057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:24.016710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.017081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.019814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.025525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:24.033960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:24.070003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e9355a0b-7062-4bef-8df1-22850dbea8c6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:24.073020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e9355a0b-7062-4bef-8df1-22850dbea8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:24.095837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 857/1725] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:24.097172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355a0b-7062-4bef-8df1-22850dbea8c6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:24.111018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355a0b-7062-4bef-8df1-22850dbea8c6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:24.116284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:24.117008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.117319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.117640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] PATH_INFO: `/v3/projects/c1f924b486d44595a251553e9542fe8a/users/9978583e383b46f79c6ad21badaa8514/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.121767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f924b486d44595a251553e9542fe8a, user_id=9978583e383b46f79c6ad21badaa8514, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:24.124710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:24.131840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 869/1726] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:23 2022] POST /identity/v3/projects => generated 405 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:24.132892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:24.138146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 858/1727] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:24 2022] PUT /identity/v3/projects/c1f924b486d44595a251553e9542fe8a/users/9978583e383b46f79c6ad21badaa8514/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:24.142178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:24.144181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:24.145419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:24.247345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b22dc2-3354-43db-87be-bbb70c954fff admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:24.255200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c1e8f-d701-49ac-b54e-816149064efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:24.256661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1e8f-d701-49ac-b54e-816149064efb admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.257191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1e8f-d701-49ac-b54e-816149064efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.257640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1e8f-d701-49ac-b54e-816149064efb admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.258999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c1e8f-d701-49ac-b54e-816149064efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:24.261543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c1e8f-d701-49ac-b54e-816149064efb admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:24.293179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:24.294322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2e663-5aa8-48fe-a8db-c540c530f766 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:24.315276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 859/1728] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:24 2022] POST /identity/v3/users => generated 309 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:24.317923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.318742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.319154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.362236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] MFA Rules not processed for user `9978583e383b46f79c6ad21badaa8514`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:24.363686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:24.417072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:24.419099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.419453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.419734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.439023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:24.442316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:24.461903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 870/1729] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:24.462715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:24.479904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82424924-7ade-4336-8725-3c277c8b6b38 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:24.488188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:24.488914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.489197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.489569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.490872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:24.494166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:24.514959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 871/1730] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:24.515941 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:24.529104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ae14-27a5-44ab-9205-4751bdbfed52 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:24.538183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 860/1731] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:24.549627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:24.551312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951b247-af89-49c1-8a9a-59b65c4793ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:24.551550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:24.552131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951b247-af89-49c1-8a9a-59b65c4793ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.552415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951b247-af89-49c1-8a9a-59b65c4793ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.552686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951b247-af89-49c1-8a9a-59b65c4793ba admin admin] PATH_INFO: `/v3/projects/6d995c0c201a48eca67b958ad10638bc/users/b61f2258d7984e5d9746f6b9dad5117b/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.578171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951b247-af89-49c1-8a9a-59b65c4793ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d995c0c201a48eca67b958ad10638bc, user_id=b61f2258d7984e5d9746f6b9dad5117b, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:24.578910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951b247-af89-49c1-8a9a-59b65c4793ba admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:24.618065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 872/1732] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:24 2022] PUT /identity/v3/projects/6d995c0c201a48eca67b958ad10638bc/users/b61f2258d7984e5d9746f6b9dad5117b/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:24.620028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1fec-4400-4a04-912c-bab594ad1128 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.620554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1fec-4400-4a04-912c-bab594ad1128 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.620970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb1fec-4400-4a04-912c-bab594ad1128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.677246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-40fb1fec-4400-4a04-912c-bab594ad1128 None None] MFA Rules not processed for user `9b8e4aad09814f8299a334639e34d948`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:24.679976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-40fb1fec-4400-4a04-912c-bab594ad1128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:24.716969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d83859-7817-4d0e-8a8d-66853d7319a6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:24.754047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:24.889987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:24.903939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:24.906481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:24.907762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:24.916238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 873/1733] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:24.917210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb1fec-4400-4a04-912c-bab594ad1128 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:25.040253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:25.066569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:25.069096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:25.081303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb1fec-4400-4a04-912c-bab594ad1128 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:25.095380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 861/1734] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:25.111363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:25.112431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:25.113016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:25.114696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:25.115853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:25.117770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:25.134547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 874/1735] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:25.138826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:25.154430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e0d34-f4be-4b75-b0b5-b1c7f0099b74 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:25.167067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:25.167760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:25.168042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:25.168319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] PATH_INFO: `/v3/projects/6d995c0c201a48eca67b958ad10638bc/users/b61f2258d7984e5d9746f6b9dad5117b/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:25.173172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d995c0c201a48eca67b958ad10638bc, user_id=b61f2258d7984e5d9746f6b9dad5117b, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:25.179260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:25.205884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:25.206016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 862/1736] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:25 2022] PUT /identity/v3/projects/6d995c0c201a48eca67b958ad10638bc/users/b61f2258d7984e5d9746f6b9dad5117b/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:25.207717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:25.249156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a0d4a-6401-49a2-9003-c75c41e56472 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:25.272242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:25.492507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb54c4-b302-4096-aa72-c3d8f242917a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:25.493739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:25.494390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:25.494662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:25.494929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] PATH_INFO: `/v3/projects/33d4ac4ff35f4130ac74c595ac69f0bc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:25.499299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d4ac4ff35f4130ac74c595ac69f0bc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:25.506561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070628bd-c426-4fd4-8297-3770cdf3e0c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:25.507648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070628bd-c426-4fd4-8297-3770cdf3e0c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:25.508453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070628bd-c426-4fd4-8297-3770cdf3e0c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:25.508733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070628bd-c426-4fd4-8297-3770cdf3e0c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:25.509368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:25.513853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-070628bd-c426-4fd4-8297-3770cdf3e0c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:25.645860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d4ac4ff35f4130ac74c595ac69f0bc', 'request_id': 'req-0b714e2d-d319-476e-931f-4a36ce326a17'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:25.657897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d4ac4ff35f4130ac74c595ac69f0bc', 'request_id': 'req-0b714e2d-d319-476e-931f-4a36ce326a17'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:25.681184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d4ac4ff35f4130ac74c595ac69f0bc', 'request_id': 'req-0b714e2d-d319-476e-931f-4a36ce326a17'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:25.692023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:25.703129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070628bd-c426-4fd4-8297-3770cdf3e0c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:25.705225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070628bd-c426-4fd4-8297-3770cdf3e0c6 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:25.719323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] The token cache is being invalidate because project 33d4ac4ff35f4130ac74c595ac69f0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:25.719625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:25.723297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0b714e2d-d319-476e-931f-4a36ce326a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:25.726203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 875/1737] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:25 2022] DELETE /identity/v3/projects/33d4ac4ff35f4130ac74c595ac69f0bc => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:25.738758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 863/1738] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:25.741508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db052e-d6a4-43c1-83a0-f28b9b5a4455 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:25.741867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db052e-d6a4-43c1-83a0-f28b9b5a4455 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:25.743488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db052e-d6a4-43c1-83a0-f28b9b5a4455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:25.862592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-20db052e-d6a4-43c1-83a0-f28b9b5a4455 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:25.864045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-20db052e-d6a4-43c1-83a0-f28b9b5a4455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:26.011786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 876/1739] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:26.019712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a2ff6-49ec-4a84-8cf4-6e56768ee3b6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.020263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a2ff6-49ec-4a84-8cf4-6e56768ee3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.020680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a2ff6-49ec-4a84-8cf4-6e56768ee3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.079488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ce1a2ff6-49ec-4a84-8cf4-6e56768ee3b6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:26.080725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ce1a2ff6-49ec-4a84-8cf4-6e56768ee3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:26.119022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 864/1740] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:26.124118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db052e-d6a4-43c1-83a0-f28b9b5a4455 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:26.153352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db052e-d6a4-43c1-83a0-f28b9b5a4455 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:26.184630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:26.185320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.185579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.185877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.187136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:26.187440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:26.189073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:26.284821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 877/1741] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:26.285882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a2ff6-49ec-4a84-8cf4-6e56768ee3b6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:26.299959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a2ff6-49ec-4a84-8cf4-6e56768ee3b6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:26.325465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990dad-b867-4392-9118-df59a093af33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:26.326295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990dad-b867-4392-9118-df59a093af33 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.326612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990dad-b867-4392-9118-df59a093af33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.326885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990dad-b867-4392-9118-df59a093af33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.327542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990dad-b867-4392-9118-df59a093af33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:26.329512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990dad-b867-4392-9118-df59a093af33 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:26.420949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 865/1742] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:26 2022] POST /identity/v3/projects => generated 393 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:26.425048 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:26.442842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b76d4-9311-413b-9cdf-e0d52ab1ac2a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:26.584842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628bfdd-caff-46ad-92c8-12de277c4feb None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.585367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628bfdd-caff-46ad-92c8-12de277c4feb None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.585806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628bfdd-caff-46ad-92c8-12de277c4feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.669651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:26.680704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.681348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.681837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.682908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:26.685724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:26.731214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 878/1743] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:26 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:26.732918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a628bfdd-caff-46ad-92c8-12de277c4feb None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:26.734860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a628bfdd-caff-46ad-92c8-12de277c4feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:26.735954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:26.755395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1a633-84e5-4e0f-9292-ac7829e14f83 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:26.772015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:26.772731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.773122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.773528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.776276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:26.778091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:26.803906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 879/1744] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:26.807916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:26.831579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71942a-ea62-4901-86d8-fcc261b6463a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:26.840058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:26.840680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.840957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.841229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.844884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:26.845078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:26.852157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:26.881342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 880/1745] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:26.884794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:26.905654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf820ed-6cc8-41f4-ac3f-831bb2b10675 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:26.921005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:26.921876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:26.922163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:26.923741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] PATH_INFO: `/v3/projects/6c7d7a5f80c547f7a55210832fefde75/users/d9cf1badc90b4e7ca2c469e34a203937/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:26.924344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 866/1746] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:26.991549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7d7a5f80c547f7a55210832fefde75, user_id=d9cf1badc90b4e7ca2c469e34a203937, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:27.002066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:27.040131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 881/1747] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:26 2022] PUT /identity/v3/projects/6c7d7a5f80c547f7a55210832fefde75/users/d9cf1badc90b4e7ca2c469e34a203937/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:27.055301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96942d96-b2fd-4367-bcca-2f25b73781d4 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:27.055826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96942d96-b2fd-4367-bcca-2f25b73781d4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:27.056174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96942d96-b2fd-4367-bcca-2f25b73781d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:27.128149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-96942d96-b2fd-4367-bcca-2f25b73781d4 None None] MFA Rules not processed for user `d9cf1badc90b4e7ca2c469e34a203937`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:27.130146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-96942d96-b2fd-4367-bcca-2f25b73781d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:27.217984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:27.219904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:27.330628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:27.381654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 867/1748] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:27.386420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96942d96-b2fd-4367-bcca-2f25b73781d4 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:27.644713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e693c6d-f874-4c3e-b8d9-a490a0aab1d8 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:27.656770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:27.658361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:27.659369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:27.659731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:27.684017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96942d96-b2fd-4367-bcca-2f25b73781d4 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:27.696238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:27.696880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:27.697164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:27.698580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:27.701093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:27.706031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:27.817244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 868/1749] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:27 2022] POST /identity/v3/projects => generated 393 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:27.821495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:27.964690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:27.973255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:28.012989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:28.027120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 882/1750] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:31:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 809 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:31:28.047499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07962416-d2af-43e9-a313-325e523ef830 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:28.062311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:28.068262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:28.069450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:28.069746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:28.070440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:28.077700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:28.192167 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:28.203417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 869/1751] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:27 2022] POST /identity/v3/users => generated 303 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:28.216255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672f867-261b-4a59-8237-621e2e2a0bd3 service swift] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:28.232137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:28.233691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:28.234100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:28.234387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:28.237246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:28.244284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:28.279087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:28.290022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 883/1752] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:28.293436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c286006-29cc-4791-8420-a034665ea872 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:28.301024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:28.302749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:28.303766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:28.303766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] PATH_INFO: `/v3/projects/f32dc4ec24da40b4903f82d00cbb998e/users/95db602db6924779b7943e4013da926f/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:28.388257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32dc4ec24da40b4903f82d00cbb998e, user_id=95db602db6924779b7943e4013da926f, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:28.409160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:28.481529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 870/1753] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:28 2022] PUT /identity/v3/projects/f32dc4ec24da40b4903f82d00cbb998e/users/95db602db6924779b7943e4013da926f/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:28.483323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:28.496520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624fbbc-005b-46fe-a42c-93762c4646c3 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:28.594227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8369556-5946-4c77-a68b-1bd19f806603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:28.596684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8369556-5946-4c77-a68b-1bd19f806603 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:28.597048 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8369556-5946-4c77-a68b-1bd19f806603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:28.598077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8369556-5946-4c77-a68b-1bd19f806603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:28.598787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8369556-5946-4c77-a68b-1bd19f806603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:28.605295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8369556-5946-4c77-a68b-1bd19f806603 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:28.642578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:28.656727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 884/1754] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:28.664484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c58a6-19b5-4f8f-8601-9b002c04425d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:28.680423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:28.681255 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:28.681787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:28.682162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] PATH_INFO: `/v3/projects/f32dc4ec24da40b4903f82d00cbb998e/users/95db602db6924779b7943e4013da926f/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:28.691497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32dc4ec24da40b4903f82d00cbb998e, user_id=95db602db6924779b7943e4013da926f, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:28.698446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:28.726419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 871/1755] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:28 2022] PUT /identity/v3/projects/f32dc4ec24da40b4903f82d00cbb998e/users/95db602db6924779b7943e4013da926f/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:28.739480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b827-10fe-48ef-bb91-3c833aff3150 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:28.739728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b827-10fe-48ef-bb91-3c833aff3150 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:28.740874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b827-10fe-48ef-bb91-3c833aff3150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:28.818916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ebb6b827-10fe-48ef-bb91-3c833aff3150 None None] MFA Rules not processed for user `95db602db6924779b7943e4013da926f`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:28.821461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ebb6b827-10fe-48ef-bb91-3c833aff3150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:28.871953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:28.968854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 885/1756] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:28.980717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:28.982544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:29.144583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62242095-b31c-4cf6-b0b5-ec4dfc88f020 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:29.168600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:29.169603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:29.170093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:29.171096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:29.195058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:29.196878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:29.218403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 872/1757] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:29.340672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14071-571f-4e96-9dae-bd505ea7d2c1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:29.341297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14071-571f-4e96-9dae-bd505ea7d2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:29.342000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14071-571f-4e96-9dae-bd505ea7d2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:29.346646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:29.348725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:29.376753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363ff28-c2b4-4739-91bd-35f6dbaac712 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:29.396557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:29.397189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:29.397498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:29.397773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:29.434246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-eed14071-571f-4e96-9dae-bd505ea7d2c1 None None] MFA Rules not processed for user `b61f2258d7984e5d9746f6b9dad5117b`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:29.435696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-eed14071-571f-4e96-9dae-bd505ea7d2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:29.511730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:29.514111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:29.543096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 873/1758] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:29.620190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 886/1759] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:30.026753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:30.028398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:30.129770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:30.174126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af5947-e271-4244-a118-013975ed4c25 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:30.181229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:30.181914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:30.182815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:30.183101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] PATH_INFO: `/v3/users/b8d777e32b98472fb09b55ef10a997e4` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:30.184400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d777e32b98472fb09b55ef10a997e4)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:30.186493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:30.262962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d777e32b98472fb09b55ef10a997e4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:30.279373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d777e32b98472fb09b55ef10a997e4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:30.298321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d777e32b98472fb09b55ef10a997e4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:30.298321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d777e32b98472fb09b55ef10a997e4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:30.300517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d777e32b98472fb09b55ef10a997e4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:30.344885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 874/1760] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:30 2022] DELETE /identity/v3/users/b8d777e32b98472fb09b55ef10a997e4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:30.349844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8fdad-6fce-49ff-8073-e971cef15037 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:30.350349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8fdad-6fce-49ff-8073-e971cef15037 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:30.350349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8fdad-6fce-49ff-8073-e971cef15037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:30.378724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6ab8fdad-6fce-49ff-8073-e971cef15037 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:30.380347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6ab8fdad-6fce-49ff-8073-e971cef15037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:30.455679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 887/1761] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:30.474479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:30.476267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:30.598000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e486835-2c48-4887-8817-f59b1e4efeb6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:30.602812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc3ef1-0def-44e8-b070-9f9cc5dd6201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:30.603849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc3ef1-0def-44e8-b070-9f9cc5dd6201 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:30.604360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc3ef1-0def-44e8-b070-9f9cc5dd6201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:30.604746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc3ef1-0def-44e8-b070-9f9cc5dd6201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:30.620594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc3ef1-0def-44e8-b070-9f9cc5dd6201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:30.622169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc3ef1-0def-44e8-b070-9f9cc5dd6201 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:30.640142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 875/1762] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:31.070610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5368cec-9b7c-4b18-b3ac-a34506924e91 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:31.070916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5368cec-9b7c-4b18-b3ac-a34506924e91 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:31.071116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5368cec-9b7c-4b18-b3ac-a34506924e91 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:31.073022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 888/1763] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:31.119525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:31.119807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:31.120281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:31.164947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:31.167437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:31.372412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 876/1764] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:31.398886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5368cec-9b7c-4b18-b3ac-a34506924e91 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:31.400648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c5368cec-9b7c-4b18-b3ac-a34506924e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:31.418435 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5368cec-9b7c-4b18-b3ac-a34506924e91 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:31.424789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:31.425501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:31.426014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:31.426284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:31.436204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:31.437853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:31.448052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 889/1765] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:31.630569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:31.632258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:31.643975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:31.651448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0d27-5a97-42ca-b6cc-9351d3161449 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:31.662371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:31.663692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:31.663989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:31.664335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] PATH_INFO: `/v3/projects/c6cb718b290042b78821ff93f238d2c0` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:31.665815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6cb718b290042b78821ff93f238d2c0)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:31.669530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:31.742199 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6cb718b290042b78821ff93f238d2c0', 'request_id': 'req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:31.749753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6cb718b290042b78821ff93f238d2c0', 'request_id': 'req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:31.764604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6cb718b290042b78821ff93f238d2c0', 'request_id': 'req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:31.791684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] The token cache is being invalidate because project c6cb718b290042b78821ff93f238d2c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:31.792640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:31.795082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-645fd7e8-553a-4ff9-a4bb-c1dcf93d033a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:31.812604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:31.813005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 877/1766] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:31 2022] DELETE /identity/v3/projects/c6cb718b290042b78821ff93f238d2c0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:31.844971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586cc8-649b-4c84-885f-973e1155d8b3 service placement] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:31.928852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:32.030310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:32.038034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:32.038639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:32.039141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] PATH_INFO: `/v3/users/9978583e383b46f79c6ad21badaa8514` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:32.042872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9978583e383b46f79c6ad21badaa8514)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:32.048390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:32.214497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9978583e383b46f79c6ad21badaa8514'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:32.226322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9978583e383b46f79c6ad21badaa8514'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:32.227725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9978583e383b46f79c6ad21badaa8514'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:32.229964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9978583e383b46f79c6ad21badaa8514'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:32.240563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a8f101a3-9798-45d0-bf96-49ae866a843f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9978583e383b46f79c6ad21badaa8514'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:32.270465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 890/1767] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:31 2022] DELETE /identity/v3/users/9978583e383b46f79c6ad21badaa8514 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:32.717779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94695-1593-4f6f-a997-edf2ade6d640 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:32.718253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94695-1593-4f6f-a997-edf2ade6d640 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:32.721338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94695-1593-4f6f-a997-edf2ade6d640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:32.804763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-49a94695-1593-4f6f-a997-edf2ade6d640 None None] MFA Rules not processed for user `8c4588df9de242e8819de0476f75959c`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:32.816582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-49a94695-1593-4f6f-a997-edf2ade6d640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:33.059477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 878/1768] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Aug 10 12:31:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:33.450116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf393e0-3872-4757-9ea7-c9f585c152ef None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:33.450700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf393e0-3872-4757-9ea7-c9f585c152ef None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:33.450700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf393e0-3872-4757-9ea7-c9f585c152ef None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:33.453157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 891/1769] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:31:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 12:31:33.475701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:33.476093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:33.476337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:33.515109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf393e0-3872-4757-9ea7-c9f585c152ef None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:33.518205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1bf393e0-3872-4757-9ea7-c9f585c152ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:33.583921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] MFA Rules not processed for user `18b7024ea0ae4deeba88ebd2f8d1f909`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:33.598047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:33.844266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 879/1770] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:33.846480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:33.849919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:33.856119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf393e0-3872-4757-9ea7-c9f585c152ef None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:33.866916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:33.867301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:33.867585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:33.867859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] PATH_INFO: `/v3/users/a22766def69f46df92a8bf657f167402` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:33.870451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22766def69f46df92a8bf657f167402)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:33.872423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:33.932219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbb0e0-7b3d-4a30-9d76-1fb300b72237 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:33.966319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:33.967514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:33.967793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:33.968102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] PATH_INFO: `/v3/projects/c1f924b486d44595a251553e9542fe8a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:33.969311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f924b486d44595a251553e9542fe8a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:33.971358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:33.994427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22766def69f46df92a8bf657f167402'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:33.997995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:34.009209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22766def69f46df92a8bf657f167402'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.010868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22766def69f46df92a8bf657f167402'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.013323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22766def69f46df92a8bf657f167402'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.040986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22766def69f46df92a8bf657f167402'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.098524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:34.156458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 892/1771] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:33 2022] DELETE /identity/v3/users/a22766def69f46df92a8bf657f167402 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:34.161146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:34.161948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f924b486d44595a251553e9542fe8a', 'request_id': 'req-63a67a72-53fb-47dc-9b8d-cdf283dd6608'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.173998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:34.180878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f924b486d44595a251553e9542fe8a', 'request_id': 'req-63a67a72-53fb-47dc-9b8d-cdf283dd6608'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.198605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f924b486d44595a251553e9542fe8a', 'request_id': 'req-63a67a72-53fb-47dc-9b8d-cdf283dd6608'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.222067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] The token cache is being invalidate because project c1f924b486d44595a251553e9542fe8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:34.222794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.227221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-63a67a72-53fb-47dc-9b8d-cdf283dd6608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:34.231262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 880/1772] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:33 2022] DELETE /identity/v3/projects/c1f924b486d44595a251553e9542fe8a => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:34.237410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27896-4bcf-45c1-a050-61bc140e91e6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:34.237699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27896-4bcf-45c1-a050-61bc140e91e6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:34.238080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27896-4bcf-45c1-a050-61bc140e91e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:34.302987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-60e27896-4bcf-45c1-a050-61bc140e91e6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:34.303467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-60e27896-4bcf-45c1-a050-61bc140e91e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:34.470246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed93c6-9771-4d9c-8009-d4499c74c5ad admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:34.487330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:34.488032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:34.488308 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:34.488587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:34.494105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:34.506126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 881/1773] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:34.519545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:34.520846 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:34.521858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:34.523822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:34.526179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:34.573630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 893/1774] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:34.575909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:34.593009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:34.639903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:34.649179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:34.704726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 882/1775] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:34.706173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:34.709906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:34.865903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83b9b6-4263-4227-997e-b8c1c22168f9 service placement] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:34.873533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:34.874167 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:34.874451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:34.874729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:34.880294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:34.936272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d182cc-a94f-4271-b8b0-5c744f3414ba None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:34.942115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:34.956382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:34.964440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:34.968884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:34.969077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:34.979339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.003776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 894/1776] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:35.007431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.014723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:35.028071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.040567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.094128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 883/1777] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:35.095008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.106776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be94e-cbe4-48ab-9279-40963b404809 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:35.129372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:35.130158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:35.130449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:35.130727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.145686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4f93-a1d8-4aa0-8e9c-f05af9df1ec4 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:35.161403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.161607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:35.163477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:35.164283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:35.165326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.166117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:35.166895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.169107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.172031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.237477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 895/1778] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:35.238616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.251889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:35.282905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 884/1779] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:35.288463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.362047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa80ff-a224-43f5-900a-e7ef64f34b86 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:35.363722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b60c-efab-4916-9736-7b3b41d62abc admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:35.380990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:35.386856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:35.387892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:35.391798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:35.392822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:35.393326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] PATH_INFO: `/v3/projects/e47542b2b1d64168b2846b18b22eadbf` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.396393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:35.396819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.398053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47542b2b1d64168b2846b18b22eadbf)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.398760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.401635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.410860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.511299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 885/1780] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:35 2022] POST /identity/v3/projects => generated 369 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:35.516381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.541885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:35.557093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:35.558531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47542b2b1d64168b2846b18b22eadbf', 'request_id': 'req-5ed31838-a728-4cde-a178-74978b1a24b0'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:35.588256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47542b2b1d64168b2846b18b22eadbf', 'request_id': 'req-5ed31838-a728-4cde-a178-74978b1a24b0'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:35.630283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47542b2b1d64168b2846b18b22eadbf', 'request_id': 'req-5ed31838-a728-4cde-a178-74978b1a24b0'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:35.676894 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] The token cache is being invalidate because project e47542b2b1d64168b2846b18b22eadbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:35.677234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:35.686301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132423-6277-411e-b35d-127f1c220c98 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:35.689754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:35.694019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 896/1781] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:35 2022] DELETE /identity/v3/projects/e47542b2b1d64168b2846b18b22eadbf => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:35.699540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.710026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:35.831820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:35.833710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:35.833991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:35.834259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.834904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.836841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.878727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed31838-a728-4cde-a178-74978b1a24b0 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:35.886253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:35.887053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:35.887681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:35.889410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] PATH_INFO: `/v3/users/b61f2258d7984e5d9746f6b9dad5117b` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.891089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61f2258d7984e5d9746f6b9dad5117b)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.893525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.893866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 886/1782] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:35 2022] POST /identity/v3/users => generated 285 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:35.894860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.899150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:35.932764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c945f-a96e-400c-acb1-03d9f6d8a08b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:35.952469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:35.953220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:35.953523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:35.953806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:35.954297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:35.956086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:35.980933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 887/1783] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:35.986588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:35.987816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61f2258d7984e5d9746f6b9dad5117b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.005102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45ef74-be97-4c66-8ecf-8e8bf048791f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.007479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61f2258d7984e5d9746f6b9dad5117b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.008329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61f2258d7984e5d9746f6b9dad5117b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.009098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61f2258d7984e5d9746f6b9dad5117b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.018001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.019227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.019514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.019789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.020449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.022115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.022775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61f2258d7984e5d9746f6b9dad5117b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.052701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 888/1784] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:36.054244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.059756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:36.071006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 897/1785] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:35 2022] DELETE /identity/v3/users/b61f2258d7984e5d9746f6b9dad5117b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:36.086135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c357-e725-475c-8b62-586a5d78ad86 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.218950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9093f-1952-4f32-b818-e8a8c347f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.219621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9093f-1952-4f32-b818-e8a8c347f277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.219916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9093f-1952-4f32-b818-e8a8c347f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.220575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9093f-1952-4f32-b818-e8a8c347f277 admin admin] PATH_INFO: `/v3/projects/6c02f5fcd3fc4586b656dd33337ef896/users/51e8538c2f024661ac8ac12a10cdebfe/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.221440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.223186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:36.237916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9093f-1952-4f32-b818-e8a8c347f277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c02f5fcd3fc4586b656dd33337ef896, user_id=51e8538c2f024661ac8ac12a10cdebfe, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.242022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9093f-1952-4f32-b818-e8a8c347f277 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.260120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 889/1786] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:36 2022] PUT /identity/v3/projects/6c02f5fcd3fc4586b656dd33337ef896/users/51e8538c2f024661ac8ac12a10cdebfe/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:36.261100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79d1d9-a903-4238-ad30-35a69cb69c1b admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.273803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe430b-1cd1-4ed4-a36e-285bf931af10 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.274406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe430b-1cd1-4ed4-a36e-285bf931af10 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.274808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe430b-1cd1-4ed4-a36e-285bf931af10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.319915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:36.324214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:36.341042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-bafe430b-1cd1-4ed4-a36e-285bf931af10 None None] MFA Rules not processed for user `51e8538c2f024661ac8ac12a10cdebfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:36.343231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.344349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.344874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.345211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-bafe430b-1cd1-4ed4-a36e-285bf931af10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:36.345674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] PATH_INFO: `/v3/users/d9cf1badc90b4e7ca2c469e34a203937` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.348066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9cf1badc90b4e7ca2c469e34a203937)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.351399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.441986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9cf1badc90b4e7ca2c469e34a203937'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.453959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9cf1badc90b4e7ca2c469e34a203937'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.454778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 890/1787] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:36.455600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9cf1badc90b4e7ca2c469e34a203937'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.457537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9cf1badc90b4e7ca2c469e34a203937'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.459738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe430b-1cd1-4ed4-a36e-285bf931af10 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.473393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d81768b-6078-4fa7-938f-702b193ed151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9cf1badc90b4e7ca2c469e34a203937'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:36.475086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe430b-1cd1-4ed4-a36e-285bf931af10 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.483727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.485540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.486720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.489246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.490175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.494498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.509603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 898/1788] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:36 2022] DELETE /identity/v3/users/d9cf1badc90b4e7ca2c469e34a203937 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:36.515962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.516259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.516689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.551239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:36.552900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:36.565692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 891/1789] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:36 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:36.571334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.578155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:36.683784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 899/1790] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:36.697846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa08b2-d690-4fdc-a309-04da6620ca5b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.703602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.704219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.704533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.704936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.705599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.707872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.718741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.720313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:36.743871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 892/1791] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:36 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:36.746206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.766308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7648d55-8398-493e-941f-3d7c85c77b8b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.773730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.778590 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.779212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.780274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.781345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.783176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.800614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd99fd6-d929-4f18-aeb0-3ea222ea2ec4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.804317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 893/1792] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:36.809075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.821917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a20e18-9d6f-45ce-b5f1-ed252cf98cd5 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:36.825003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:36.828589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.829226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.829565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.829864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] PATH_INFO: `/v3/projects/a74dd28897b64ebb81655e5aa66699aa/users/08fb5f9735d248d587e717de3a38948c/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.847163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74dd28897b64ebb81655e5aa66699aa, user_id=08fb5f9735d248d587e717de3a38948c, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.850431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.875572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 894/1793] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:36 2022] PUT /identity/v3/projects/a74dd28897b64ebb81655e5aa66699aa/users/08fb5f9735d248d587e717de3a38948c/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:36.877148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.904898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:36.905554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:36.905873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:36.908153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:36.924760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:36.930010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:36.955246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 900/1794] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:36.955529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:36.976644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:36.978245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:36.997507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e250e-d0cf-4dcd-bb58-4cbe4fd9ef5c service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:37.004295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:37.004971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.005217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.005519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] PATH_INFO: `/v3/projects/6d995c0c201a48eca67b958ad10638bc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.007147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d995c0c201a48eca67b958ad10638bc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:37.009877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:37.022220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708eb853-2e3f-4f4e-920e-e94696073229 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:37.033056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:37.033837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.034114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.034386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.035768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:37.038967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:37.067263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 895/1795] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:37.068502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:37.090197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0897e-1aac-4922-957e-1f698384994d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:37.112803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d995c0c201a48eca67b958ad10638bc', 'request_id': 'req-ce56de5a-2eb5-4054-99af-8e14d03d9536'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:37.148440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d995c0c201a48eca67b958ad10638bc', 'request_id': 'req-ce56de5a-2eb5-4054-99af-8e14d03d9536'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:37.168258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d995c0c201a48eca67b958ad10638bc', 'request_id': 'req-ce56de5a-2eb5-4054-99af-8e14d03d9536'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:37.207215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] The token cache is being invalidate because project 6d995c0c201a48eca67b958ad10638bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:37.207790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:37.215552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce56de5a-2eb5-4054-99af-8e14d03d9536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:37.217649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 901/1796] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:36 2022] DELETE /identity/v3/projects/6d995c0c201a48eca67b958ad10638bc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:37.222161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96d51-b6f3-446d-879f-32c0696e6e6f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.222807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96d51-b6f3-446d-879f-32c0696e6e6f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.223189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96d51-b6f3-446d-879f-32c0696e6e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.224564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce2350-739d-410e-8d5c-1c3a8a185e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:37.225405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2350-739d-410e-8d5c-1c3a8a185e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.225688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2350-739d-410e-8d5c-1c3a8a185e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.226117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce2350-739d-410e-8d5c-1c3a8a185e02 admin admin] PATH_INFO: `/v3/projects/a74dd28897b64ebb81655e5aa66699aa/users/08fb5f9735d248d587e717de3a38948c/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.228757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce2350-739d-410e-8d5c-1c3a8a185e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74dd28897b64ebb81655e5aa66699aa, user_id=08fb5f9735d248d587e717de3a38948c, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:37.233518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce2350-739d-410e-8d5c-1c3a8a185e02 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:37.255022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-f7a96d51-b6f3-446d-879f-32c0696e6e6f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:37.256664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f7a96d51-b6f3-446d-879f-32c0696e6e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.265820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 896/1797] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:37 2022] PUT /identity/v3/projects/a74dd28897b64ebb81655e5aa66699aa/users/08fb5f9735d248d587e717de3a38948c/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:37.278257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ec7ce-ffe8-4cb1-8901-13a49850a5b0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.279159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ec7ce-ffe8-4cb1-8901-13a49850a5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.279472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ec7ce-ffe8-4cb1-8901-13a49850a5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.349805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1a8ec7ce-ffe8-4cb1-8901-13a49850a5b0 None None] MFA Rules not processed for user `08fb5f9735d248d587e717de3a38948c`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:37.351325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1a8ec7ce-ffe8-4cb1-8901-13a49850a5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.373884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 902/1798] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:37.375401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9fe8-ceb9-4189-9778-05837e3e3a58 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.375800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9fe8-ceb9-4189-9778-05837e3e3a58 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.376185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9fe8-ceb9-4189-9778-05837e3e3a58 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.377841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 903/1799] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:37.382173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3f9c2-0fa2-441e-82c1-6d740d573368 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.382565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3f9c2-0fa2-441e-82c1-6d740d573368 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.382948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3f9c2-0fa2-441e-82c1-6d740d573368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.408673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-74c3f9c2-0fa2-441e-82c1-6d740d573368 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:37.410521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-74c3f9c2-0fa2-441e-82c1-6d740d573368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.423193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 904/1800] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:37.424674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be79f84-e2ff-4a62-b492-e89c08d21227 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.425407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be79f84-e2ff-4a62-b492-e89c08d21227 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.425812 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be79f84-e2ff-4a62-b492-e89c08d21227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.454726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-3be79f84-e2ff-4a62-b492-e89c08d21227 None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:37.456080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3be79f84-e2ff-4a62-b492-e89c08d21227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.484686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 897/1801] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:37.485866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ec7ce-ffe8-4cb1-8901-13a49850a5b0 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:37.507755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ec7ce-ffe8-4cb1-8901-13a49850a5b0 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:37.512792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:37.513435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.513741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.514158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.514813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:37.515902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:37.517616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:37.523677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:37.535953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 898/1802] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:37.540946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:37.546002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.560991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 905/1803] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:37.563090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d4a70-6ed1-4236-ad4e-749bbb78196a None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.563539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d4a70-6ed1-4236-ad4e-749bbb78196a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.563960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d4a70-6ed1-4236-ad4e-749bbb78196a None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.566273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 906/1804] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:37.567163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d4a70-6ed1-4236-ad4e-749bbb78196a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:37.583421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d4a70-6ed1-4236-ad4e-749bbb78196a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:37.588458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:37.588961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.589240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.589712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.590491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:37.596194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:37.627968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:37.630560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 907/1805] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:37 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:37.634653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:37.636698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.648753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bc783-939d-4379-9804-15a159fe2fb9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:37.714150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:37.714771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.715048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.715319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.732580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1914da2-68ac-4894-b583-a57a8155707f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:37.739038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8514bb-d85f-4748-a05b-65923e3581a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:37.739750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8514bb-d85f-4748-a05b-65923e3581a6 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.740095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8514bb-d85f-4748-a05b-65923e3581a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.740437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8514bb-d85f-4748-a05b-65923e3581a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.749828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8514bb-d85f-4748-a05b-65923e3581a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:37.751520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8514bb-d85f-4748-a05b-65923e3581a6 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:37.769645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 908/1806] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:37.770847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e132079-1fe7-446d-b6df-9c726d479f9a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:37.771127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e132079-1fe7-446d-b6df-9c726d479f9a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:37.771393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e132079-1fe7-446d-b6df-9c726d479f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:37.794640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:37.796455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5e132079-1fe7-446d-b6df-9c726d479f9a None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:37.797506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:37.799119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5e132079-1fe7-446d-b6df-9c726d479f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.811278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 909/1807] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:37.812240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e132079-1fe7-446d-b6df-9c726d479f9a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:37.823878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 899/1808] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:37.958512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:37.960172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:37.999302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e132079-1fe7-446d-b6df-9c726d479f9a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.005499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.006264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.006993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.006993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.007655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.009745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.048815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 910/1809] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:37 2022] POST /identity/v3/users => generated 283 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:38.053930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.075410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92997368-ee36-45bd-af19-fd05f8a26746 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.089902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.090586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.091160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.091432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.092648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.094434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.106309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 911/1810] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:38.107618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.121240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e510a52-62d1-4c0a-81bf-5238914172d4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.127211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.127974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.128254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.128526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.145948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.148386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.157037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48ff76-d5dc-41e5-a152-7e9c9ab026c0 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.158673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:38.163372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.164042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.164324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.164726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] PATH_INFO: `/v3/projects/6c7d7a5f80c547f7a55210832fefde75` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.165838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7d7a5f80c547f7a55210832fefde75)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.168970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.179134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 912/1811] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:38.180186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.192273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5f4a6-86f1-4e2f-9819-1bd5ea2262c2 service nova] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.196938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.197736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.198091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.198428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.199125 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.201119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.221031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 913/1812] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:38.222042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.230247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:38.233532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6817d8-d48c-40e7-882a-b47c9da088db admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.237486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7d7a5f80c547f7a55210832fefde75', 'request_id': 'req-aa4bfc48-f59b-41d9-9505-3049b0e3e866'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.245139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7d7a5f80c547f7a55210832fefde75', 'request_id': 'req-aa4bfc48-f59b-41d9-9505-3049b0e3e866'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.254619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7d7a5f80c547f7a55210832fefde75', 'request_id': 'req-aa4bfc48-f59b-41d9-9505-3049b0e3e866'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.272453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] The token cache is being invalidate because project 6c7d7a5f80c547f7a55210832fefde75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:38.272732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.274323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.275963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 900/1813] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:37 2022] DELETE /identity/v3/projects/6c7d7a5f80c547f7a55210832fefde75 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:38.281527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.283947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:38.299304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9614e31-e4e6-42d7-92eb-38cb22472534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.299972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9614e31-e4e6-42d7-92eb-38cb22472534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.300274 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9614e31-e4e6-42d7-92eb-38cb22472534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.300670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9614e31-e4e6-42d7-92eb-38cb22472534 admin admin] PATH_INFO: `/v3/projects/3c14ef87f5414af1956159ad43de33db/users/aa5308c1f4af4f0488b9d0bbc3ebd6f1/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.318873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9614e31-e4e6-42d7-92eb-38cb22472534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c14ef87f5414af1956159ad43de33db, user_id=aa5308c1f4af4f0488b9d0bbc3ebd6f1, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.321804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9614e31-e4e6-42d7-92eb-38cb22472534 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.337995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 914/1814] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:38 2022] PUT /identity/v3/projects/3c14ef87f5414af1956159ad43de33db/users/aa5308c1f4af4f0488b9d0bbc3ebd6f1/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:38.353495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.353785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.354161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.375949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bfc48-f59b-41d9-9505-3049b0e3e866 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.383433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] MFA Rules not processed for user `aa5308c1f4af4f0488b9d0bbc3ebd6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:38.384687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:38.453031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.453771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.454122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.454466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] PATH_INFO: `/v3/users/95db602db6924779b7943e4013da926f` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.455791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95db602db6924779b7943e4013da926f)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.458240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.497403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 915/1815] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:38.501968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.505248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:38.561971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a44d4-ddfe-46e4-b091-812919f990c5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.571455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.572084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.572750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.572750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.573343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.575849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.605971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95db602db6924779b7943e4013da926f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.623661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 916/1816] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:38 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:38.631375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.631555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95db602db6924779b7943e4013da926f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.639415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95db602db6924779b7943e4013da926f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.640302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95db602db6924779b7943e4013da926f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.640992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95db602db6924779b7943e4013da926f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:38.662891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:38.673608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 901/1817] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:38 2022] DELETE /identity/v3/users/95db602db6924779b7943e4013da926f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:38.706994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:38.730224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e6870-92bb-435a-b3c6-11abb00dac8f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.812490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.813211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.813589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.813860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.815393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.818204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.859025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 917/1818] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:38 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:38.862322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.866985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:38.891254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a431d6-1484-41aa-b2a1-1eea512881f4 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.932877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.933681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.934040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.934226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:38.934971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:38.940534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:38.958534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 902/1819] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:38.962925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:38.989507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020156-f1f7-4387-9d59-9b513cd03195 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:38.994855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:38.995770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:38.996245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:38.996696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] PATH_INFO: `/v3/projects/ca77263d538b4dfe9fa30047fa5b40d2/users/555d4eca68444ead8e191f780f511cee/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:39.023010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca77263d538b4dfe9fa30047fa5b40d2, user_id=555d4eca68444ead8e191f780f511cee, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:39.028127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:39.064109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 918/1820] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:38 2022] PUT /identity/v3/projects/ca77263d538b4dfe9fa30047fa5b40d2/users/555d4eca68444ead8e191f780f511cee/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:39.069470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:39.094037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e26025-b6f6-4476-b009-7a979e3b9a60 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:39.216227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240f8e2-7599-4de2-bab6-3716380df46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:39.217996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240f8e2-7599-4de2-bab6-3716380df46a admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:39.218607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240f8e2-7599-4de2-bab6-3716380df46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:39.219051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240f8e2-7599-4de2-bab6-3716380df46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:39.219944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240f8e2-7599-4de2-bab6-3716380df46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:39.221769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240f8e2-7599-4de2-bab6-3716380df46a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:39.234479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 903/1821] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:39.240282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:39.255617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42d6c1-6fc0-470b-b581-ed680a9d48ce admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:39.275521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:39.276204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:39.276481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:39.276755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] PATH_INFO: `/v3/projects/ca77263d538b4dfe9fa30047fa5b40d2/users/555d4eca68444ead8e191f780f511cee/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:39.282560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca77263d538b4dfe9fa30047fa5b40d2, user_id=555d4eca68444ead8e191f780f511cee, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:39.285791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:39.320428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 919/1822] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:39 2022] PUT /identity/v3/projects/ca77263d538b4dfe9fa30047fa5b40d2/users/555d4eca68444ead8e191f780f511cee/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:39.332210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:39.332501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:39.332781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:39.418202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] MFA Rules not processed for user `555d4eca68444ead8e191f780f511cee`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:39.419713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:39.497420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:39.499681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:39.698651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde3d94-15ba-4e42-a219-02de4a776da6 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:39.703251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 904/1823] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:39.709224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:39.709923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:39.710200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:39.710468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] PATH_INFO: `/v3/projects/f32dc4ec24da40b4903f82d00cbb998e` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:39.711661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32dc4ec24da40b4903f82d00cbb998e)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:39.714839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:39.754920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:39.757049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:39.825136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:39.844879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32dc4ec24da40b4903f82d00cbb998e', 'request_id': 'req-a51a0e16-9534-4393-9e5f-847e09421cf8'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:39.870846 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32dc4ec24da40b4903f82d00cbb998e', 'request_id': 'req-a51a0e16-9534-4393-9e5f-847e09421cf8'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:39.900180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32dc4ec24da40b4903f82d00cbb998e', 'request_id': 'req-a51a0e16-9534-4393-9e5f-847e09421cf8'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:39.901444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3f558-88af-4f14-b0dd-3eea30c0ad62 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:39.912567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] The token cache is being invalidate because project f32dc4ec24da40b4903f82d00cbb998e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:39.913153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:39.914060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a51a0e16-9534-4393-9e5f-847e09421cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:39.916038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 920/1824] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:39 2022] DELETE /identity/v3/projects/f32dc4ec24da40b4903f82d00cbb998e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:39.921790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25ad9f-762a-4a60-82a3-9ee1ec8fb2df None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:39.922814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25ad9f-762a-4a60-82a3-9ee1ec8fb2df None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:39.928266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25ad9f-762a-4a60-82a3-9ee1ec8fb2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:39.974717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:39.976266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-4c25ad9f-762a-4a60-82a3-9ee1ec8fb2df None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:39.977950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4c25ad9f-762a-4a60-82a3-9ee1ec8fb2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:40.037700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.038350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.040082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.040380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.047354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:40.154083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 921/1825] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:40.161323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052433be-1e74-47d8-adf0-ee1c375999c5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.161566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052433be-1e74-47d8-adf0-ee1c375999c5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.161851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052433be-1e74-47d8-adf0-ee1c375999c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.190516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.191818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-052433be-1e74-47d8-adf0-ee1c375999c5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:40.193010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.193124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-052433be-1e74-47d8-adf0-ee1c375999c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:40.204823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 922/1826] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:40.206791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052433be-1e74-47d8-adf0-ee1c375999c5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:40.218049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 905/1827] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:40.222669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052433be-1e74-47d8-adf0-ee1c375999c5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:40.231191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.231820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.232153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.232369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.233733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:40.234042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.235683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.283678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 923/1828] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:40.288176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:40.315486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2461d0-03fa-4483-95b5-b5104bdb5747 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:40.328392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.329258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.329662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.330930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.331753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.334951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.429468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 906/1829] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:40 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:40.433710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:40.459373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f720260-4cca-4c7e-a82c-f17819f13a5a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:40.562400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:40.564914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:40.598932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.599616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.599896 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.600273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.600799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.603068 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.641323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 924/1830] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:40 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:40.646347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:40.690945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95377-8089-4592-95c0-78466c276106 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:40.697429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.698222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.698583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.699082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.699823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.702390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.722603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 925/1831] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:40.727616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:40.746949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2e150-5028-42b4-b8a2-c4d3e858f7f4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:40.765871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.765871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.765871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.765871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.766326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.766326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.790837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60b0a2-7667-4c56-9616-424846e82a3f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:40.794690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 926/1832] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:40.803200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:40.819404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.821992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.822400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.822652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] PATH_INFO: `/v3/users/51e8538c2f024661ac8ac12a10cdebfe` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.827908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e8538c2f024661ac8ac12a10cdebfe)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.832604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.844644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3785ea-430d-4bda-94e5-2a6473e20d57 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:40.869252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d5a84-144b-4120-a369-628dcb60d408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:40.869999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d5a84-144b-4120-a369-628dcb60d408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:40.870287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d5a84-144b-4120-a369-628dcb60d408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:40.872210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d5a84-144b-4120-a369-628dcb60d408 admin admin] PATH_INFO: `/v3/projects/529dd19021ea42de9f0a5c4252ef24a0/users/c5e076950fce43a0a17ff2bed4cedfd3/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:40.911086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:40.922643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d5a84-144b-4120-a369-628dcb60d408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=529dd19021ea42de9f0a5c4252ef24a0, user_id=c5e076950fce43a0a17ff2bed4cedfd3, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:40.938989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d5a84-144b-4120-a369-628dcb60d408 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:40.961906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e8538c2f024661ac8ac12a10cdebfe'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:40.977295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 927/1833] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:40 2022] PUT /identity/v3/projects/529dd19021ea42de9f0a5c4252ef24a0/users/c5e076950fce43a0a17ff2bed4cedfd3/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:40.991704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e8538c2f024661ac8ac12a10cdebfe'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:40.996559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:41.001040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:41.001626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:41.002331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:41.022841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e8538c2f024661ac8ac12a10cdebfe'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.024234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e8538c2f024661ac8ac12a10cdebfe'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.024947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e8b95908-53bf-42f6-9df4-f7869734b16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e8538c2f024661ac8ac12a10cdebfe'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.046511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] MFA Rules not processed for user `c5e076950fce43a0a17ff2bed4cedfd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:41.048004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.073611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 907/1834] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:40 2022] DELETE /identity/v3/users/51e8538c2f024661ac8ac12a10cdebfe => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:41.078955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:41.080048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:41.080431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:41.137554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:41.141788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.266749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 908/1835] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:41.268425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:41.275130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.281523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 928/1836] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:41.286276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:41.287934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.319453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed0f53-7b7d-43a6-ae52-a7edbe7ff934 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:41.324543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:41.325360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:41.325538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:41.325945 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] PATH_INFO: `/v3/users/aa5308c1f4af4f0488b9d0bbc3ebd6f1` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:41.328000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5308c1f4af4f0488b9d0bbc3ebd6f1)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:41.329969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:41.465314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d503-ce4b-47cd-97dd-1e8b6097a9ac None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:41.468787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5308c1f4af4f0488b9d0bbc3ebd6f1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.480817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c30fb-f717-460c-b74f-e1537910ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:41.481531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c30fb-f717-460c-b74f-e1537910ddad admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:41.482401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c30fb-f717-460c-b74f-e1537910ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:41.482401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c30fb-f717-460c-b74f-e1537910ddad admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:41.482578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5308c1f4af4f0488b9d0bbc3ebd6f1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.483853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c30fb-f717-460c-b74f-e1537910ddad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:41.493837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c30fb-f717-460c-b74f-e1537910ddad admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:41.496981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5308c1f4af4f0488b9d0bbc3ebd6f1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.499956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5308c1f4af4f0488b9d0bbc3ebd6f1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.500930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5308c1f4af4f0488b9d0bbc3ebd6f1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:41.539286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 909/1837] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:41 2022] DELETE /identity/v3/users/aa5308c1f4af4f0488b9d0bbc3ebd6f1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:41.540564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:41.542219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.589480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 929/1838] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:41 2022] POST /identity/v3/projects => generated 383 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:41.591650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:41.594260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:41.594748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:41.645643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:41.646948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.706455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:41.733153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c6b6f-d1d9-4250-b818-034fc4f873bc admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:41.839291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 930/1839] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:41.847164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:41.855392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.904058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c527356-eff0-4993-b713-005dc82d3164 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:41.906873 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:41.908887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:41.909174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:41.910927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:41.915002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:41.918690 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:41.920595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:41.920922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:41.921160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:41.923001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:41.928481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:41.951826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:41.953463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:41.955336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:41.983007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 931/1840] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:41 2022] POST /identity/v3/users => generated 298 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:41.984139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:41.999041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 910/1841] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:42.002652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:42.008033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1db0b-d833-4475-9b80-878780b7dcfa admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:42.022034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:42.023015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:42.023330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:42.023560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:42.044341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73473cbf-ded6-4e34-85ee-26570204e49d service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:42.055571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:42.057639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:42.069228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:42.074193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:42.074536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:42.074797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:42.078802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:42.081117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 932/1842] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:42.094740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:42.131479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 911/1843] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:42.135108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:42.158526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e286671-c668-4dc3-8d10-a044ef7205dc service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:42.178507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:42.182881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:42.185350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:42.185789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] PATH_INFO: `/v3/projects/05f4510d907b43a09e46bc8b58818f87/users/f1a2f5046aba4a71ae4267e1b6a46bfd/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:42.245108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f4510d907b43a09e46bc8b58818f87, user_id=f1a2f5046aba4a71ae4267e1b6a46bfd, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:42.248916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:42.271934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 933/1844] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:42 2022] PUT /identity/v3/projects/05f4510d907b43a09e46bc8b58818f87/users/f1a2f5046aba4a71ae4267e1b6a46bfd/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:42.277947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:42.303689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424925d-e151-4ecd-bc9b-f9db4ae2b2d2 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:42.468040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c30211-ff77-434b-b5b1-d7b075717c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:42.468664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30211-ff77-434b-b5b1-d7b075717c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:42.469097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30211-ff77-434b-b5b1-d7b075717c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:42.469440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30211-ff77-434b-b5b1-d7b075717c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:42.470139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c30211-ff77-434b-b5b1-d7b075717c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:42.471836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c30211-ff77-434b-b5b1-d7b075717c2a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:42.489321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:42.504630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 912/1845] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:42.506873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448393c-8aa0-462c-aaf1-f7c98c8f5406 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:42.512031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:42.512789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:42.521661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:42.522178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] PATH_INFO: `/v3/projects/05f4510d907b43a09e46bc8b58818f87/users/f1a2f5046aba4a71ae4267e1b6a46bfd/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:42.525401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f4510d907b43a09e46bc8b58818f87, user_id=f1a2f5046aba4a71ae4267e1b6a46bfd, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:42.528353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:42.569231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:42.570028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:42.570028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:42.585296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 934/1846] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:42 2022] PUT /identity/v3/projects/05f4510d907b43a09e46bc8b58818f87/users/f1a2f5046aba4a71ae4267e1b6a46bfd/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:42.630250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] MFA Rules not processed for user `f1a2f5046aba4a71ae4267e1b6a46bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:42.633407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:42.788501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 913/1847] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:42.807553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:42.809995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:42.959333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b747d-763a-4358-93ee-c62a9e218c70 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:42.964211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:42.965018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:42.965454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:42.966205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:43.003138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:43.004756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:43.038893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 935/1848] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:43.310018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:43.311696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:43.355324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccba22f-de02-4e5a-a59f-33c384cb7fd5 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:43.368997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc5def-2e80-48c5-8646-a0176003b92a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:43.369979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc5def-2e80-48c5-8646-a0176003b92a service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:43.370404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc5def-2e80-48c5-8646-a0176003b92a service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:43.371868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc5def-2e80-48c5-8646-a0176003b92a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:43.376645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-eddc5def-2e80-48c5-8646-a0176003b92a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:43.438268 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:43.523029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc5def-2e80-48c5-8646-a0176003b92a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:43.524722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc5def-2e80-48c5-8646-a0176003b92a service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:43.545696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 914/1849] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:44.333812 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:44.335379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:44.510041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c98b14-1ffa-4e20-81cf-102c2daaffff service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:44.524708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:44.525842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:44.526529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:44.526994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] PATH_INFO: `/v3/users/57c9881ba9f94c86bae34ceb4ab30924` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:44.528570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c9881ba9f94c86bae34ceb4ab30924)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:44.531410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:44.642074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:INSERT=5,keystone:SELECT=89,keystone:DELETE=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:44.646976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c9881ba9f94c86bae34ceb4ab30924'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:44.666302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c9881ba9f94c86bae34ceb4ab30924'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:44.667532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c9881ba9f94c86bae34ceb4ab30924'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:44.668858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c9881ba9f94c86bae34ceb4ab30924'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:44.691091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c9881ba9f94c86bae34ceb4ab30924'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:44.751509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 936/1850] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:44 2022] DELETE /identity/v3/users/57c9881ba9f94c86bae34ceb4ab30924 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:44.755935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:44.756466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:44.756466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:44.769834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:44.771539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:44.781139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:44.782614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:44.926265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 915/1851] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:44.927753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:44.929660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:44.957465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314bf05-446b-4d4d-af53-39ea241230d3 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:44.966651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:44.967755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:44.968478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:44.969129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] PATH_INFO: `/v3/projects/3c14ef87f5414af1956159ad43de33db` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:44.970661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c14ef87f5414af1956159ad43de33db)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:44.975575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:44.978156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccd7a9-1668-40aa-a685-804a9edc894f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:44.991243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:44.991895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:44.992178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:44.992452 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] PATH_INFO: `/v3/projects/6c02f5fcd3fc4586b656dd33337ef896` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:44.993666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c02f5fcd3fc4586b656dd33337ef896)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:44.995984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:45.060571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c14ef87f5414af1956159ad43de33db', 'request_id': 'req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.066626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c02f5fcd3fc4586b656dd33337ef896', 'request_id': 'req-374cfa0c-d700-46a9-a228-8b4109df357c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.068896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c14ef87f5414af1956159ad43de33db', 'request_id': 'req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.080099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c14ef87f5414af1956159ad43de33db', 'request_id': 'req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.082037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c02f5fcd3fc4586b656dd33337ef896', 'request_id': 'req-374cfa0c-d700-46a9-a228-8b4109df357c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.094045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] The token cache is being invalidate because project 3c14ef87f5414af1956159ad43de33db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:45.094415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.095089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.096268 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c02f5fcd3fc4586b656dd33337ef896', 'request_id': 'req-374cfa0c-d700-46a9-a228-8b4109df357c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.096828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 916/1852] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:44 2022] DELETE /identity/v3/projects/3c14ef87f5414af1956159ad43de33db => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:45.111806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:45.117947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:45.120340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] The token cache is being invalidate because project 6c02f5fcd3fc4586b656dd33337ef896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:45.120735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.121811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.124014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 937/1853] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:44 2022] DELETE /identity/v3/projects/6c02f5fcd3fc4586b656dd33337ef896 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:45.125791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:45.127551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:45.210644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:45.224815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:45.253236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eeb65-5f4d-4c8a-9c2d-654550a81ae7 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:45.257777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cfa0c-d700-46a9-a228-8b4109df357c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:45.274086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:45.275193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:45.275664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:45.276287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] PATH_INFO: `/v3/users/555d4eca68444ead8e191f780f511cee` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:45.278081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=555d4eca68444ead8e191f780f511cee)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:45.281388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:45.346782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555d4eca68444ead8e191f780f511cee'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.356680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:45.357436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:45.357780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:45.358139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:45.359033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555d4eca68444ead8e191f780f511cee'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.360511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555d4eca68444ead8e191f780f511cee'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.360625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:45.361743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555d4eca68444ead8e191f780f511cee'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.371469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555d4eca68444ead8e191f780f511cee'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.377852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:45.379586 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:45.391445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 917/1854] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:45.392427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:45.393833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:45.412804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3da9f-04a5-45f4-8777-d8fcc5d89f17 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:45.416905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 938/1855] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:45 2022] DELETE /identity/v3/users/555d4eca68444ead8e191f780f511cee => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:45.418610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:45.419520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:45.420531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:45.421020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] PATH_INFO: `/v3/users/08fb5f9735d248d587e717de3a38948c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:45.422388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08fb5f9735d248d587e717de3a38948c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:45.424585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:45.510136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08fb5f9735d248d587e717de3a38948c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.522363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08fb5f9735d248d587e717de3a38948c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.533526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08fb5f9735d248d587e717de3a38948c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.534724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08fb5f9735d248d587e717de3a38948c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.535450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08fb5f9735d248d587e717de3a38948c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:45.577616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 918/1856] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:45 2022] DELETE /identity/v3/users/08fb5f9735d248d587e717de3a38948c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:46.135152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:46.136845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.219874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:46.222867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.251519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2f41-bdef-47bb-a3cd-c4899ec05948 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:46.340260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:46.341186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.341650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.342207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] PATH_INFO: `/v3/projects/6db6bc4e043a4cde8d3c63cf84a1122f` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.343611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db6bc4e043a4cde8d3c63cf84a1122f)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:46.346642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:46.363585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25d44-00ab-4bf6-998b-4950a3a8ade3 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:46.368615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:46.369252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.369557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.369912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] PATH_INFO: `/v3/projects/ca77263d538b4dfe9fa30047fa5b40d2` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.370937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca77263d538b4dfe9fa30047fa5b40d2)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:46.372856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:46.380290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:46.391038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:46.423657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db6bc4e043a4cde8d3c63cf84a1122f', 'request_id': 'req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.430075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db6bc4e043a4cde8d3c63cf84a1122f', 'request_id': 'req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.442192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db6bc4e043a4cde8d3c63cf84a1122f', 'request_id': 'req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.459010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] The token cache is being invalidate because project 6db6bc4e043a4cde8d3c63cf84a1122f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:46.459380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.460220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.461959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 939/1857] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:46 2022] DELETE /identity/v3/projects/6db6bc4e043a4cde8d3c63cf84a1122f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:46.462849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:46.464185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.474459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca77263d538b4dfe9fa30047fa5b40d2', 'request_id': 'req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.479949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca77263d538b4dfe9fa30047fa5b40d2', 'request_id': 'req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.488145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca77263d538b4dfe9fa30047fa5b40d2', 'request_id': 'req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.499932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] The token cache is being invalidate because project ca77263d538b4dfe9fa30047fa5b40d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:46.500216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.500846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.502556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 919/1858] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:46 2022] DELETE /identity/v3/projects/ca77263d538b4dfe9fa30047fa5b40d2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:46.503117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:46.504375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.538428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fd47b-85ee-4e0f-b75d-1187dc799e89 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:46.574920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737cdee-0ab5-455c-ad1e-afce59c5bd5b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:46.586012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:46.586654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.587014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.587192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] PATH_INFO: `/v3/users/0f708b08f6d44c4fb3e7344f36eadcad` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.588385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f708b08f6d44c4fb3e7344f36eadcad)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:46.590359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:46.600987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:46.601721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.602489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.602489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] PATH_INFO: `/v3/projects/a74dd28897b64ebb81655e5aa66699aa` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.603386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74dd28897b64ebb81655e5aa66699aa)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:46.606240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:46.648457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f708b08f6d44c4fb3e7344f36eadcad'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.659009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f708b08f6d44c4fb3e7344f36eadcad'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.659371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74dd28897b64ebb81655e5aa66699aa', 'request_id': 'req-a613b456-8303-438a-9e1b-e0f1c37949b5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.665081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:46.667115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74dd28897b64ebb81655e5aa66699aa', 'request_id': 'req-a613b456-8303-438a-9e1b-e0f1c37949b5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.669469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f708b08f6d44c4fb3e7344f36eadcad'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.671153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f708b08f6d44c4fb3e7344f36eadcad'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.672143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f708b08f6d44c4fb3e7344f36eadcad'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.677132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74dd28897b64ebb81655e5aa66699aa', 'request_id': 'req-a613b456-8303-438a-9e1b-e0f1c37949b5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.688927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] The token cache is being invalidate because project a74dd28897b64ebb81655e5aa66699aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:46.689216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.689935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a613b456-8303-438a-9e1b-e0f1c37949b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.691663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 940/1859] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:46 2022] DELETE /identity/v3/projects/a74dd28897b64ebb81655e5aa66699aa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:46.693602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 920/1860] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:46 2022] DELETE /identity/v3/users/0f708b08f6d44c4fb3e7344f36eadcad => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:46.694396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:46.695778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.695952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31197f-14b3-4bbe-898c-bc2db24cd242 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.696244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31197f-14b3-4bbe-898c-bc2db24cd242 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.696525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31197f-14b3-4bbe-898c-bc2db24cd242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.717702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ec31197f-14b3-4bbe-898c-bc2db24cd242 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:46.719789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ec31197f-14b3-4bbe-898c-bc2db24cd242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.723240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:46.801349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 941/1861] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:46.801547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbb340-aeb5-4c56-ac1f-8b863b6b3449 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:46.804099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358da91b-0952-4dc2-b6bc-7d57cc4f9b19 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.804390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358da91b-0952-4dc2-b6bc-7d57cc4f9b19 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.805408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358da91b-0952-4dc2-b6bc-7d57cc4f9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.806323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:46.807032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.807385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.807672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] PATH_INFO: `/v3/users/c5e076950fce43a0a17ff2bed4cedfd3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.808887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e076950fce43a0a17ff2bed4cedfd3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:46.811020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:46.830046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-358da91b-0952-4dc2-b6bc-7d57cc4f9b19 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:46.831382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-358da91b-0952-4dc2-b6bc-7d57cc4f9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.840776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 942/1862] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:46.842582 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1450-f767-4d00-bba1-30c85b309584 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.842864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1450-f767-4d00-bba1-30c85b309584 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.843144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1450-f767-4d00-bba1-30c85b309584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.867389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6f8b1450-f767-4d00-bba1-30c85b309584 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:46.868721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6f8b1450-f767-4d00-bba1-30c85b309584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.885337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 943/1863] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:46.887198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:46.887624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:46.887966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:46.890552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e076950fce43a0a17ff2bed4cedfd3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.900010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e076950fce43a0a17ff2bed4cedfd3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.910157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e076950fce43a0a17ff2bed4cedfd3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.910990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e076950fce43a0a17ff2bed4cedfd3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.911781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e076950fce43a0a17ff2bed4cedfd3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:46.912612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:46.914977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.924859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 944/1864] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:46.926135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:46.928155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:46.938205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 921/1865] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:46 2022] DELETE /identity/v3/users/c5e076950fce43a0a17ff2bed4cedfd3 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:46.938205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:46.948726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2df21a-c8ef-4e52-a67d-c9c3cf14e7f2 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.021093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba21da-c4fc-471e-81dd-d48abe30c625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.021843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba21da-c4fc-471e-81dd-d48abe30c625 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.022251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba21da-c4fc-471e-81dd-d48abe30c625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.022596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba21da-c4fc-471e-81dd-d48abe30c625 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.023526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba21da-c4fc-471e-81dd-d48abe30c625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:47.023920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba21da-c4fc-471e-81dd-d48abe30c625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.025951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba21da-c4fc-471e-81dd-d48abe30c625 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.038653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc99f8-cc7e-46eb-84b9-f3507be8689f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.040132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 945/1866] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:47.041990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc92135-b896-4e8a-ae62-cf73a7b09d80 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.042360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc92135-b896-4e8a-ae62-cf73a7b09d80 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.042708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc92135-b896-4e8a-ae62-cf73a7b09d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.046772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.047606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.048077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.048522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.049468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:47.049946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.053470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.064209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9dc92135-b896-4e8a-ae62-cf73a7b09d80 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:47.065730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9dc92135-b896-4e8a-ae62-cf73a7b09d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:47.072409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 922/1867] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:47.074115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.079120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 946/1868] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:47.080174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc92135-b896-4e8a-ae62-cf73a7b09d80 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.086173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d15cca-a5cb-4828-8789-7c299eae4db9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.093028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc92135-b896-4e8a-ae62-cf73a7b09d80 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.093159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.093685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.093963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.094371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.095007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.098374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.099801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.100639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.101140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.102454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.103463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.105521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.140934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 923/1869] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:47.142100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.145759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:47.165345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:47.187224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 947/1870] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:47.187224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.240545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:47.258383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088d9ca-d6db-41f4-8691-74742fe51c83 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.289327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c05f9-670b-47b4-afd0-a69a152c0046 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.294772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.295638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.296134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.296621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.297491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.299640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.329921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 948/1871] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/users => generated 282 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:47.331839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.343727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.344698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.345376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.345803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.350062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf2a08-8673-423b-a6cc-a3b7fc6f5140 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.357552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.358585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.359075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.359558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.360713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.364392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.368207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.370706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.387082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 924/1872] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:47.388261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.398651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 949/1873] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/users => generated 287 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:47.401094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea8e07-e505-4310-9699-b86bf01e1feb service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.403248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.407612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.408229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.408509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.408786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.416572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.419006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.420923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbb7b8-ee62-42d0-b11a-7035a89f5b5e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.428882 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.430217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.430680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.431164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.432053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.434029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 925/1874] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:47.436809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.438774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.450558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da5173-0aab-4a45-a6be-eb1e5b69fb7e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.453101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 950/1875] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:47.457653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.457653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.457653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.457653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.458378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.459503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.460022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.475430 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a449c7b-2c24-4552-9bae-c147520f097d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.475572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 926/1876] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:47.476339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.481057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.481764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.483906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.484800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.485928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.488072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.490797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6cb8d-c5a6-4b05-97cb-620611fc1050 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.497224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522c595-a3ee-4301-b34a-40adb6991525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.498901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522c595-a3ee-4301-b34a-40adb6991525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.499864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522c595-a3ee-4301-b34a-40adb6991525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.500343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522c595-a3ee-4301-b34a-40adb6991525 admin admin] PATH_INFO: `/v3/projects/d271c89bf34a43fcacae50251155e897/users/03b625aeaaaa41d5b77d68298fd11e84/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.502714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 951/1877] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:47.507084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.519583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522c595-a3ee-4301-b34a-40adb6991525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d271c89bf34a43fcacae50251155e897, user_id=03b625aeaaaa41d5b77d68298fd11e84, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.522465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522c595-a3ee-4301-b34a-40adb6991525 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.526238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c504-f198-42be-bd67-6cc51aef01d4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.532960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d7332-8acc-4184-ba67-ec54887c9a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.534047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d7332-8acc-4184-ba67-ec54887c9a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.534708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d7332-8acc-4184-ba67-ec54887c9a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.535198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d7332-8acc-4184-ba67-ec54887c9a12 admin admin] PATH_INFO: `/v3/projects/88e9de0d802d4d34999965553195d83a/users/e949aad6ddf240bb964e0cb3bcd776ce/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.539517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 927/1878] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:47 2022] PUT /identity/v3/projects/d271c89bf34a43fcacae50251155e897/users/03b625aeaaaa41d5b77d68298fd11e84/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:47.551301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e028a77-440c-46bf-a1d5-0f37e18357b8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.551650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e028a77-440c-46bf-a1d5-0f37e18357b8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.551859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e028a77-440c-46bf-a1d5-0f37e18357b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.555607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d7332-8acc-4184-ba67-ec54887c9a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e9de0d802d4d34999965553195d83a, user_id=e949aad6ddf240bb964e0cb3bcd776ce, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.559395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d7332-8acc-4184-ba67-ec54887c9a12 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.600749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 952/1879] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:47 2022] PUT /identity/v3/projects/88e9de0d802d4d34999965553195d83a/users/e949aad6ddf240bb964e0cb3bcd776ce/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:47.602684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274676e-3f9b-4dfd-9b34-e171c07e06d1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.603903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274676e-3f9b-4dfd-9b34-e171c07e06d1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.604413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274676e-3f9b-4dfd-9b34-e171c07e06d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.608397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6e028a77-440c-46bf-a1d5-0f37e18357b8 None None] MFA Rules not processed for user `03b625aeaaaa41d5b77d68298fd11e84`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:47.611693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6e028a77-440c-46bf-a1d5-0f37e18357b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:47.643102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9274676e-3f9b-4dfd-9b34-e171c07e06d1 None None] MFA Rules not processed for user `e949aad6ddf240bb964e0cb3bcd776ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:47.644464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9274676e-3f9b-4dfd-9b34-e171c07e06d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:47.713769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 928/1880] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:47.718332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e028a77-440c-46bf-a1d5-0f37e18357b8 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.736176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:47.753130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 953/1881] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:47.758273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274676e-3f9b-4dfd-9b34-e171c07e06d1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.814635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e028a77-440c-46bf-a1d5-0f37e18357b8 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.824329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.825365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.825671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.826032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.826897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.828976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.837248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:47.860603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274676e-3f9b-4dfd-9b34-e171c07e06d1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:47.866958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:47.868315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:47.868906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:47.869470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:47.870714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:47.874292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:47.874413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 929/1882] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:47.875275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.912124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 954/1883] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:47.914021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:47.988507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ae544-c5e8-440d-bb2f-a99b9ffb03e0 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.085126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a569-e12a-415c-84e6-4912d3156f47 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.110501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.113163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.114050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.114421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.115031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.135488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.168565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 955/1884] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:48.171361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.173999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.174700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.174985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.175259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.176093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.179071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.180106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.195822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00605669-2ee1-4360-ad83-3eb2e5412a63 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.203318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.204878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.205240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.205665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] PATH_INFO: `/v3/projects/aad22d1b83ef48a984a1e1a2fd00b761` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.207292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad22d1b83ef48a984a1e1a2fd00b761)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.209591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.220794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 930/1885] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:47 2022] POST /identity/v3/users => generated 289 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:48.221234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.233955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0e026-e77e-4630-980a-5fe739349023 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.239864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.240650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.240937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.241209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.241898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.244130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.264220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 931/1886] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:48.264220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.274870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c6924-bd7f-460d-b2f6-8f0dbbe42e02 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.275639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad22d1b83ef48a984a1e1a2fd00b761', 'request_id': 'req-6cef0d1f-35d4-46db-ad44-6884d0b56082'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.286049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad22d1b83ef48a984a1e1a2fd00b761', 'request_id': 'req-6cef0d1f-35d4-46db-ad44-6884d0b56082'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.296213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad22d1b83ef48a984a1e1a2fd00b761', 'request_id': 'req-6cef0d1f-35d4-46db-ad44-6884d0b56082'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.299690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:48.312301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] The token cache is being invalidate because project aad22d1b83ef48a984a1e1a2fd00b761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:48.312647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.313377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.317352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 956/1887] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:48 2022] DELETE /identity/v3/projects/aad22d1b83ef48a984a1e1a2fd00b761 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:48.318467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.322092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.367034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2d723-4503-44f8-9f4f-99542136f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.367592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2d723-4503-44f8-9f4f-99542136f4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.367873 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2d723-4503-44f8-9f4f-99542136f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.368146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2d723-4503-44f8-9f4f-99542136f4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.368793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2d723-4503-44f8-9f4f-99542136f4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.370512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2d723-4503-44f8-9f4f-99542136f4c4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.380980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 932/1888] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:48.382349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.382636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.382910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.402679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:48.406316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:48.407785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.419140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 933/1889] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:48.426535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef0d1f-35d4-46db-ad44-6884d0b56082 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.428015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.429516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.433516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6809aa8-b6bd-4302-8bd8-bb7268f1c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.434198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6809aa8-b6bd-4302-8bd8-bb7268f1c051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.434479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6809aa8-b6bd-4302-8bd8-bb7268f1c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.434754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6809aa8-b6bd-4302-8bd8-bb7268f1c051 admin admin] PATH_INFO: `/v3/projects/3c2a7d47db0c4daa9420317a845ed726/users/2356d5adf1344e2b939e50ac730c4c22/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.441982 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbb55e-96ba-4d11-901f-ff3b2cf73068 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.449442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.450222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.450502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.450775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] PATH_INFO: `/v3/projects/0a0c13bef6e84147833f79d13fc259ee/users/bf9a0e04494146f6a01f003005458506/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.454549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6809aa8-b6bd-4302-8bd8-bb7268f1c051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2a7d47db0c4daa9420317a845ed726, user_id=2356d5adf1344e2b939e50ac730c4c22, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.457333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6809aa8-b6bd-4302-8bd8-bb7268f1c051 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.469168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0c13bef6e84147833f79d13fc259ee, user_id=bf9a0e04494146f6a01f003005458506, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.472681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.477350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 957/1890] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:48 2022] PUT /identity/v3/projects/3c2a7d47db0c4daa9420317a845ed726/users/2356d5adf1344e2b939e50ac730c4c22/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:48.479222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4e45b-c43b-4126-b21a-8bddae21d7d3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.479511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4e45b-c43b-4126-b21a-8bddae21d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.480181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4e45b-c43b-4126-b21a-8bddae21d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.491542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 934/1891] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:48 2022] PUT /identity/v3/projects/0a0c13bef6e84147833f79d13fc259ee/users/bf9a0e04494146f6a01f003005458506/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:48.492727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.494494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.503838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-1cb4e45b-c43b-4126-b21a-8bddae21d7d3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:48.505131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1cb4e45b-c43b-4126-b21a-8bddae21d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.571655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 958/1892] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:48.572327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4e45b-c43b-4126-b21a-8bddae21d7d3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.573084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666476d2-bc09-478d-9cfd-b695034ac01b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.578293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.578939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.579218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.579895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] PATH_INFO: `/v3/projects/529dd19021ea42de9f0a5c4252ef24a0` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.580885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=529dd19021ea42de9f0a5c4252ef24a0)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.582836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.590306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4e45b-c43b-4126-b21a-8bddae21d7d3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.601361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.601987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.602264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.602536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.603187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.604800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.615441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 959/1893] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:48.616351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.628929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f918-4758-428e-9a82-f41c3ae30581 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.645066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '529dd19021ea42de9f0a5c4252ef24a0', 'request_id': 'req-d58ddeb8-b604-4d1c-897f-3b3688eb4133'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.651062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '529dd19021ea42de9f0a5c4252ef24a0', 'request_id': 'req-d58ddeb8-b604-4d1c-897f-3b3688eb4133'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.663730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '529dd19021ea42de9f0a5c4252ef24a0', 'request_id': 'req-d58ddeb8-b604-4d1c-897f-3b3688eb4133'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.683452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] The token cache is being invalidate because project 529dd19021ea42de9f0a5c4252ef24a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:48.684126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.685791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.688218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 935/1894] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:48 2022] DELETE /identity/v3/projects/529dd19021ea42de9f0a5c4252ef24a0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:48.690188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.692951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.698171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.698864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.699214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.699588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.700310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.702679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.707910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ddeb8-b604-4d1c-897f-3b3688eb4133 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.712979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.713621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.713901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.714175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.714824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:48.715303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.715721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 960/1895] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:48.717251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.717842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.719340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.731279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 936/1896] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:48.732266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.733980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.734243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa95c34-f225-44f1-b3c3-93849f1d17e4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.741071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.742006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.742301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.742579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] PATH_INFO: `/v3/projects/3c2a7d47db0c4daa9420317a845ed726/users/2356d5adf1344e2b939e50ac730c4c22/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.744798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:48.747298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2a7d47db0c4daa9420317a845ed726, user_id=2356d5adf1344e2b939e50ac730c4c22, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.747514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489b38c-5cfe-4ff8-976b-a536996c8055 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.753778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.753907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.754409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.754693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.755030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] PATH_INFO: `/v3/users/f1a2f5046aba4a71ae4267e1b6a46bfd` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.756322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a2f5046aba4a71ae4267e1b6a46bfd)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.758556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.770650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 961/1897] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:48 2022] PUT /identity/v3/projects/3c2a7d47db0c4daa9420317a845ed726/users/2356d5adf1344e2b939e50ac730c4c22/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:48.771208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.773356 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.818594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a2f5046aba4a71ae4267e1b6a46bfd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.821681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:48.827238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a2f5046aba4a71ae4267e1b6a46bfd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.835526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a2f5046aba4a71ae4267e1b6a46bfd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.836406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a2f5046aba4a71ae4267e1b6a46bfd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.837115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a2f5046aba4a71ae4267e1b6a46bfd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:48.846588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0b68d-14e5-4005-939c-313133fda22d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.851466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb71d69-97b6-426a-9aa2-1476a3421594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:48.852123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71d69-97b6-426a-9aa2-1476a3421594 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.852406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71d69-97b6-426a-9aa2-1476a3421594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.852692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71d69-97b6-426a-9aa2-1476a3421594 admin admin] PATH_INFO: `/v3/projects/0a0c13bef6e84147833f79d13fc259ee/users/bf9a0e04494146f6a01f003005458506/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.854611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb71d69-97b6-426a-9aa2-1476a3421594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0c13bef6e84147833f79d13fc259ee, user_id=bf9a0e04494146f6a01f003005458506, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:48.854802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 937/1898] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:48 2022] DELETE /identity/v3/users/f1a2f5046aba4a71ae4267e1b6a46bfd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:48.855579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:48.856887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.857428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb71d69-97b6-426a-9aa2-1476a3421594 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:48.868728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 962/1899] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:48 2022] PUT /identity/v3/projects/0a0c13bef6e84147833f79d13fc259ee/users/bf9a0e04494146f6a01f003005458506/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:48.870139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080b42-b351-49eb-906d-32e42dae93fc None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.870421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080b42-b351-49eb-906d-32e42dae93fc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.870695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080b42-b351-49eb-906d-32e42dae93fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:48.901474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-26080b42-b351-49eb-906d-32e42dae93fc None None] MFA Rules not processed for user `2356d5adf1344e2b939e50ac730c4c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:48.902684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-26080b42-b351-49eb-906d-32e42dae93fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:48.942331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad87d-c40f-4ec2-ad3d-164b0d9ee826 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:48.984692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 963/1900] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:48.986174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eadb3-8e44-4c4a-ac4d-498350b9a6e0 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:48.986486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eadb3-8e44-4c4a-ac4d-498350b9a6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:48.986742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eadb3-8e44-4c4a-ac4d-498350b9a6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.021159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.022278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.022569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.022852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.023498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.025901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.044678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-eb9eadb3-8e44-4c4a-ac4d-498350b9a6e0 None None] MFA Rules not processed for user `bf9a0e04494146f6a01f003005458506`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:49.046375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-eb9eadb3-8e44-4c4a-ac4d-498350b9a6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.066967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 938/1901] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:48 2022] POST /identity/v3/projects => generated 387 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:49.067811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.072580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.084258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:49.151927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729e097-1d29-4417-ad24-8e5cd4c681e1 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.158650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.159331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.159557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.159837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.162587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.181998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 964/1902] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:49.183863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9eadb3-8e44-4c4a-ac4d-498350b9a6e0 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.203375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9eadb3-8e44-4c4a-ac4d-498350b9a6e0 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.256079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:49.270838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.272339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.278240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.279192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.279570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.279972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.280709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.282883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 939/1903] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:49.283918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.284107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.285469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.299442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf201-56a5-4ee6-a253-48d76ccc7a0a service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.305132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.306014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.306451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.306832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.316087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 965/1904] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:49 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:49.320576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.334104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63947677-4809-4305-bfd3-0a8c490a04ad admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.339500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.340609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.341069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.341592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.342618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.345014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.360425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 966/1905] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:49.364908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.381839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246afcee-1c47-48c3-ad9b-e11c0418e375 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.387745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.389303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.390843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.391878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.392505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.393540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.394605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.398487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.401806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 940/1906] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:49.412993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 967/1907] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:49.417376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.432179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c7b5c-3e50-4c64-bf22-3ba23a9761b8 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.438287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134c25a-9615-495f-a046-9cc85c09d549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.439282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c25a-9615-495f-a046-9cc85c09d549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.439928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c25a-9615-495f-a046-9cc85c09d549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.440263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c25a-9615-495f-a046-9cc85c09d549 admin admin] PATH_INFO: `/v3/projects/e100286925174f36b3260211cb564c5c/users/d02183bbf039429395a69fe0259a34cc/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.475772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134c25a-9615-495f-a046-9cc85c09d549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e100286925174f36b3260211cb564c5c, user_id=d02183bbf039429395a69fe0259a34cc, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.479130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134c25a-9615-495f-a046-9cc85c09d549 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.490107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.492695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.501424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 941/1908] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:49 2022] PUT /identity/v3/projects/e100286925174f36b3260211cb564c5c/users/d02183bbf039429395a69fe0259a34cc/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:49.508580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5bfc-df30-49fd-80c1-54fd320fefd8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.509019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5bfc-df30-49fd-80c1-54fd320fefd8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.509140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5bfc-df30-49fd-80c1-54fd320fefd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.534951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cfd3f-3582-453c-9f6c-ff961f7fbdf0 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.577200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-cafb5bfc-df30-49fd-80c1-54fd320fefd8 None None] MFA Rules not processed for user `d02183bbf039429395a69fe0259a34cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:49.577990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cafb5bfc-df30-49fd-80c1-54fd320fefd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.610661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:49.688066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 942/1909] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:49.688500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.689149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.689491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5bfc-df30-49fd-80c1-54fd320fefd8 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.689603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.689761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] PATH_INFO: `/v3/projects/05f4510d907b43a09e46bc8b58818f87` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.691135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f4510d907b43a09e46bc8b58818f87)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.693649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.769907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f4510d907b43a09e46bc8b58818f87', 'request_id': 'req-7f873565-44db-40c1-b035-17d08f7d83bb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:49.780128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f4510d907b43a09e46bc8b58818f87', 'request_id': 'req-7f873565-44db-40c1-b035-17d08f7d83bb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:49.793175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5bfc-df30-49fd-80c1-54fd320fefd8 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.797746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f4510d907b43a09e46bc8b58818f87', 'request_id': 'req-7f873565-44db-40c1-b035-17d08f7d83bb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:49.810810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] The token cache is being invalidate because project 05f4510d907b43a09e46bc8b58818f87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:49.811178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:49.812051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:49.817313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 968/1910] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:49 2022] DELETE /identity/v3/projects/05f4510d907b43a09e46bc8b58818f87 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:49.818208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.819719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.877485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:49.878666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.879372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.879643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.879909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.881962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.897380 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f873565-44db-40c1-b035-17d08f7d83bb admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:49.905974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc9fcd-9b74-4707-9a08-028a46571165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:49.906891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9fcd-9b74-4707-9a08-028a46571165 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.907364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9fcd-9b74-4707-9a08-028a46571165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.907641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9fcd-9b74-4707-9a08-028a46571165 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.908334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc9fcd-9b74-4707-9a08-028a46571165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.910359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc9fcd-9b74-4707-9a08-028a46571165 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.938923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 969/1911] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:49 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:49.940282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122a232-07e9-491f-b795-8f76e6393045 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:49.940574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122a232-07e9-491f-b795-8f76e6393045 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:49.940851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122a232-07e9-491f-b795-8f76e6393045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:49.959842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9122a232-07e9-491f-b795-8f76e6393045 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:49.960230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:49.961042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9122a232-07e9-491f-b795-8f76e6393045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:49.961717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:49.971133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 943/1912] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:49.971907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:49.982081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023881a-fba8-4eb5-8fa1-5ccf24731390 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.013039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:50.033681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 970/1913] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:50.042429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24350ae1-8edb-43b2-a5b2-f78de382fe5a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.042429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24350ae1-8edb-43b2-a5b2-f78de382fe5a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.042704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24350ae1-8edb-43b2-a5b2-f78de382fe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.055122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.055746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.056026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.056308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.056940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.058921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.065356 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-24350ae1-8edb-43b2-a5b2-f78de382fe5a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:50.066924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-24350ae1-8edb-43b2-a5b2-f78de382fe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:50.081336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 971/1914] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:50.082077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 944/1915] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:49 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:50.085252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24350ae1-8edb-43b2-a5b2-f78de382fe5a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.090477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.095913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24350ae1-8edb-43b2-a5b2-f78de382fe5a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.101675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.102643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.102833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949dfe-f645-492a-a329-9467c7b02341 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.103086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.103468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.104196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:50.104702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.106538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.108743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.109404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.109839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.110132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.110834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.112619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.122588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 972/1916] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:31:50.124636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 945/1917] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:50.128364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.128578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.139470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232adb7a-d43f-495e-8a85-5f5f80296f20 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.144327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.144929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.145210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.145514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.146030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b85e71-eb56-429a-bf91-9677464de961 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.146145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.148162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.150719 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.151319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.151602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.151878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.152741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.155283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.167802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 946/1918] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:50.171767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.177709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 973/1919] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:50.181693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe06774-026b-49ae-9ed2-c320b117df30 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.181822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.243476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.244145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.244426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.244703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] PATH_INFO: `/v3/projects/aee1b86c998f4e67870f0311913b9a91/users/4f7dbdc2447143a4958e6376f466f566/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.248351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07353c4-c20d-4aa6-a5ff-ed67e0623824 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.253102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f25ef-c30a-422a-9e95-c5a4fc31cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.253774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f25ef-c30a-422a-9e95-c5a4fc31cdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.254055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f25ef-c30a-422a-9e95-c5a4fc31cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.254328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f25ef-c30a-422a-9e95-c5a4fc31cdc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.254960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f25ef-c30a-422a-9e95-c5a4fc31cdc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.257040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f25ef-c30a-422a-9e95-c5a4fc31cdc1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.260200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee1b86c998f4e67870f0311913b9a91, user_id=4f7dbdc2447143a4958e6376f466f566, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.263154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.277797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 947/1920] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:50 2022] PUT /identity/v3/projects/aee1b86c998f4e67870f0311913b9a91/users/4f7dbdc2447143a4958e6376f466f566/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:50.282318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 974/1921] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:50.286411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.292507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5edef-47f6-44d2-9409-add1aff8b5c3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.292819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5edef-47f6-44d2-9409-add1aff8b5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.293119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5edef-47f6-44d2-9409-add1aff8b5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.328647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-3ab5edef-47f6-44d2-9409-add1aff8b5c3 None None] MFA Rules not processed for user `4f7dbdc2447143a4958e6376f466f566`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:50.329957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3ab5edef-47f6-44d2-9409-add1aff8b5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:50.334133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:50.368410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad27f8-dc28-4594-bed9-e2fe92ca86b9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.374220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.374834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.375114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.375389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.376038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.377683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.389871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 948/1922] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:50.402278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.412332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658b1a2-b013-4ca1-9666-f9f9a1e6d975 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.416818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 975/1923] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:50.417339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.417834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.418117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.418392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.419032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.420666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.421393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5edef-47f6-44d2-9409-add1aff8b5c3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.432258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5edef-47f6-44d2-9409-add1aff8b5c3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.432534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 949/1924] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:50.437196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.438369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.438979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.439262 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.439531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.440242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.442492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.447253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeade0d-02d3-4800-8908-f5359001899e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.452532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d96cb5-a49a-446e-9d67-cc71369d78f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.453176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96cb5-a49a-446e-9d67-cc71369d78f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.453488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96cb5-a49a-446e-9d67-cc71369d78f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.453765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96cb5-a49a-446e-9d67-cc71369d78f1 admin admin] PATH_INFO: `/v3/projects/da37cde3ab2e49c38f99d0f5c9ba6a54/users/d1ff5b6b63da412f8981c711da67c647/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.456893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:50.470438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d96cb5-a49a-446e-9d67-cc71369d78f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da37cde3ab2e49c38f99d0f5c9ba6a54, user_id=d1ff5b6b63da412f8981c711da67c647, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.472561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 976/1925] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:50.474258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d96cb5-a49a-446e-9d67-cc71369d78f1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.476865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.488041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 950/1926] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:50 2022] PUT /identity/v3/projects/da37cde3ab2e49c38f99d0f5c9ba6a54/users/d1ff5b6b63da412f8981c711da67c647/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:50.499564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f951-ad57-42ce-a80e-63757757060f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.499852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f951-ad57-42ce-a80e-63757757060f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.500133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f951-ad57-42ce-a80e-63757757060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.534318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e6a2f951-ad57-42ce-a80e-63757757060f None None] MFA Rules not processed for user `d1ff5b6b63da412f8981c711da67c647`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:50.535699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e6a2f951-ad57-42ce-a80e-63757757060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:50.560121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ef678-55bd-44a4-8ef2-c63b8eb60eba admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.636379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 951/1927] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:50.641781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f951-ad57-42ce-a80e-63757757060f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.665178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.666361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.666770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.667069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.667970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.671938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.726360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 977/1928] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/users => generated 299 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:50.727641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.734635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f951-ad57-42ce-a80e-63757757060f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.740653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.741312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.741552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd4fd0-30f1-44bc-a9a1-1156667265af admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.741735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.742026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.742827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.745167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.747442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.748759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.749047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.749576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.750388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.752089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.768575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 978/1929] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:50.770331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.781883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce9e4-27a0-41be-ac74-73e7f29fe621 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.782431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 952/1930] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:50.787453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:50.797135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:50.912531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.913952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.915276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.915794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] PATH_INFO: `/v3/projects/dae150012d1146fd92212b8a9d72e15f/users/8609691780d2420ca5228c4e154736fd/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.931605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce2efb-31a5-4cb4-8947-23bde7c2dd06 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:50.941476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:50.943856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:50.944400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:50.944681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:50.945344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.947719 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.961626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae150012d1146fd92212b8a9d72e15f, user_id=8609691780d2420ca5228c4e154736fd, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:50.964891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:50.969238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:50.993747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 979/1931] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:50 2022] PUT /identity/v3/projects/dae150012d1146fd92212b8a9d72e15f/users/8609691780d2420ca5228c4e154736fd/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:50.998490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 953/1932] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:50 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:50.998836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.004653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.025697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d5832-09ab-4df0-97be-5f1d8c4e65ef admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:51.142784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf865ea9-27d8-4e25-bda8-0c679340dd41 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:51.154676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42499efa-8e3f-4595-9d78-a490daa549a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:51.157878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42499efa-8e3f-4595-9d78-a490daa549a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.158176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42499efa-8e3f-4595-9d78-a490daa549a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.159098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42499efa-8e3f-4595-9d78-a490daa549a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.161127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42499efa-8e3f-4595-9d78-a490daa549a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:51.162883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42499efa-8e3f-4595-9d78-a490daa549a1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:51.187754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace58136-7f87-4b9a-acc2-7efbb55fede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:51.188454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace58136-7f87-4b9a-acc2-7efbb55fede0 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.190130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace58136-7f87-4b9a-acc2-7efbb55fede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.190420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace58136-7f87-4b9a-acc2-7efbb55fede0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.191552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace58136-7f87-4b9a-acc2-7efbb55fede0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:51.192535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 980/1933] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:51.194581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccd27-12b0-4340-b1c2-add822437137 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.194870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccd27-12b0-4340-b1c2-add822437137 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.195162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccd27-12b0-4340-b1c2-add822437137 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.196625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace58136-7f87-4b9a-acc2-7efbb55fede0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:51.199431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 981/1934] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:31:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:31:51.200233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ccd27-12b0-4340-b1c2-add822437137 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.216281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 954/1935] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:51.229777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f02b6-641a-45c3-8b55-92733c5d425d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.230897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f02b6-641a-45c3-8b55-92733c5d425d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.231341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f02b6-641a-45c3-8b55-92733c5d425d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.236368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ccd27-12b0-4340-b1c2-add822437137 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:51.247511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:51.248108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.261496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.261496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] PATH_INFO: `/v3/projects/dae150012d1146fd92212b8a9d72e15f/users/8609691780d2420ca5228c4e154736fd/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.261496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae150012d1146fd92212b8a9d72e15f, user_id=8609691780d2420ca5228c4e154736fd, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:51.261983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:51.273005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-cf4f02b6-641a-45c3-8b55-92733c5d425d None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:51.274273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cf4f02b6-641a-45c3-8b55-92733c5d425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:51.295318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 982/1936] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:51 2022] PUT /identity/v3/projects/dae150012d1146fd92212b8a9d72e15f/users/8609691780d2420ca5228c4e154736fd/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:51.295318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.383186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 955/1937] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:51.385338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.385809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.386277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.398781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30b1c-9583-4d13-9803-5171bf17d192 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:51.405954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:51.408168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.408459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.408734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.410594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:51.412755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:51.440013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 983/1938] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:51.440013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.444519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:51.465140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] MFA Rules not processed for user `8609691780d2420ca5228c4e154736fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:51.468506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:51.478867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f979f-791e-428c-8b45-071927e2c633 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:51.494532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:51.495441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.495441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.495718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.514734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:51.516456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:51.521530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:51.531148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 984/1939] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:51.532940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.552708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89809a13-b313-406e-9f94-f3163b5d0f1d service nova] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:51.561723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff8b32-bfb3-440d-a6c3-9003651bfdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:51.562942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8b32-bfb3-440d-a6c3-9003651bfdad admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.563499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8b32-bfb3-440d-a6c3-9003651bfdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.564020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8b32-bfb3-440d-a6c3-9003651bfdad admin admin] PATH_INFO: `/v3/projects/7b62d92a1ac94cf28fda666f6b60657e/users/3623113d27824ec9bcda950e2da0ec01/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.589177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff8b32-bfb3-440d-a6c3-9003651bfdad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b62d92a1ac94cf28fda666f6b60657e, user_id=3623113d27824ec9bcda950e2da0ec01, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:51.594939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff8b32-bfb3-440d-a6c3-9003651bfdad admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:51.642785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 985/1940] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:51 2022] PUT /identity/v3/projects/7b62d92a1ac94cf28fda666f6b60657e/users/3623113d27824ec9bcda950e2da0ec01/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:51.653631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f6844-2ab2-4e33-b4b6-aacf2d4a97ba None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.654755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f6844-2ab2-4e33-b4b6-aacf2d4a97ba None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.655201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f6844-2ab2-4e33-b4b6-aacf2d4a97ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:51.670114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 956/1941] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:51.682912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.696028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:51.710016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:51.726168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9f5f6844-2ab2-4e33-b4b6-aacf2d4a97ba None None] MFA Rules not processed for user `3623113d27824ec9bcda950e2da0ec01`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:51.732722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9f5f6844-2ab2-4e33-b4b6-aacf2d4a97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:51.877778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328484f-8ab5-42f4-ac4a-8960b8c8d54b None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:51.921094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 986/1942] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:51.922032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f6844-2ab2-4e33-b4b6-aacf2d4a97ba None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:51.952811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:51.954130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:51.954455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:51.954729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.024653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f6844-2ab2-4e33-b4b6-aacf2d4a97ba None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.029583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.030203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.030491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.030755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.031389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.033371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.036031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.037515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.047908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 957/1943] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:52.069166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 987/1944] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:51 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:52.070298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.082960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e637b42-3a27-4367-acec-257dc856799d service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.147758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.148377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.148656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.149456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.150123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.152065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.170986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:52.179738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 958/1945] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:52 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:31:52.190309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.201469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4be8f9-a8d9-4b43-96f1-b7c28a57c5d1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.206396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.207123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.207403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.207676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.208328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.209961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.226852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 988/1946] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:52.226967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.236730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b799659-a30a-48f7-b29c-3b4c6c4eac63 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.241454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.242133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.242429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.242709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] PATH_INFO: `/v3/projects/268e8eaa712b4e0cb78cde7ad9a3d01c/users/6671e6a9672340d5915879ae1914ef67/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.259181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268e8eaa712b4e0cb78cde7ad9a3d01c, user_id=6671e6a9672340d5915879ae1914ef67, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.262181 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.277585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 959/1947] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:52 2022] PUT /identity/v3/projects/268e8eaa712b4e0cb78cde7ad9a3d01c/users/6671e6a9672340d5915879ae1914ef67/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:52.281919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.293078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a28454-ccc4-406c-991d-8e55083408f0 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.311658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.313531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:52.336997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:52.375478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.376152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.376383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.376663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.377346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.378967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.396589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 989/1948] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:52.397391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.409226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009fcbd-86d8-4543-9fdc-b54252957be9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.411634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7ec4-6e4f-4ff3-9db6-6e7f7888c590 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.415738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.416370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.416659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.416935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] PATH_INFO: `/v3/users/e949aad6ddf240bb964e0cb3bcd776ce` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.418578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84f365-993c-4f82-b98c-2f787ffe6261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.419336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f365-993c-4f82-b98c-2f787ffe6261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.420206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f365-993c-4f82-b98c-2f787ffe6261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.420457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e949aad6ddf240bb964e0cb3bcd776ce)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.421316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f365-993c-4f82-b98c-2f787ffe6261 admin admin] PATH_INFO: `/v3/projects/268e8eaa712b4e0cb78cde7ad9a3d01c/users/6671e6a9672340d5915879ae1914ef67/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.422941 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84f365-993c-4f82-b98c-2f787ffe6261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268e8eaa712b4e0cb78cde7ad9a3d01c, user_id=6671e6a9672340d5915879ae1914ef67, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.423597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.425958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84f365-993c-4f82-b98c-2f787ffe6261 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.441311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 990/1949] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:52 2022] PUT /identity/v3/projects/268e8eaa712b4e0cb78cde7ad9a3d01c/users/6671e6a9672340d5915879ae1914ef67/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:52.451640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.452126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.452624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.499051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] MFA Rules not processed for user `6671e6a9672340d5915879ae1914ef67`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:52.500611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:52.508674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e949aad6ddf240bb964e0cb3bcd776ce'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:52.521370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e949aad6ddf240bb964e0cb3bcd776ce'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:52.533398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e949aad6ddf240bb964e0cb3bcd776ce'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:52.534142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e949aad6ddf240bb964e0cb3bcd776ce'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:52.535019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-87792059-17cd-4c97-8987-168eb0a01855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e949aad6ddf240bb964e0cb3bcd776ce'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:52.558019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 960/1950] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:52 2022] DELETE /identity/v3/users/e949aad6ddf240bb964e0cb3bcd776ce => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:52.562311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.562604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.562882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.584111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:52.585925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:52.608828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 991/1951] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:52.616363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.626140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:52.649250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:52.680833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 961/1952] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:52.690379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bdb85-8e4f-49df-9f55-914484d38573 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.694480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.695986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:52.767420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.768051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.768334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.768608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.784084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:52.788266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb0e6a-0cb6-4646-baa0-79aa36dec1bf None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.799416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.800468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.800975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.801634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.814883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.816996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.832381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 962/1953] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:52.834141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:52.843097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:52.850886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:52.852033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:52.862305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabec14-b202-4ba4-bd1b-6cb7e1b57af5 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:52.868062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:52.869196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:52.869758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:52.870158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:52.874051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:52.883099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 992/1954] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:53.016499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:53.020570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:53.038549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 963/1955] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:53.044743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:53.054209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef42a07-ffb6-4460-a11a-d103f256a7a3 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:53.071193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:53.071844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:53.078901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:53.079523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:53.086127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:53.202590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:53.204795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:53.215847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 993/1956] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:53.655894 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:53.657089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:53.675706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb399eb-e839-4ddf-bd75-da738687ad4d service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:53.681567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:53.682212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:53.682501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:53.682836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] PATH_INFO: `/v3/projects/88e9de0d802d4d34999965553195d83a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:53.686052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e9de0d802d4d34999965553195d83a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:53.686886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:53.785129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e9de0d802d4d34999965553195d83a', 'request_id': 'req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:53.795377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e9de0d802d4d34999965553195d83a', 'request_id': 'req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:53.810951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e9de0d802d4d34999965553195d83a', 'request_id': 'req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:53.825403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] The token cache is being invalidate because project 88e9de0d802d4d34999965553195d83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:53.825728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:53.826504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:53.829250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 964/1957] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:53 2022] DELETE /identity/v3/projects/88e9de0d802d4d34999965553195d83a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:53.833230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:53.851546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c838a5-adf4-4247-8a06-8a7bbc3eb823 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:53.946406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:53.947310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:53.947793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:53.948174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] PATH_INFO: `/v3/users/2356d5adf1344e2b939e50ac730c4c22` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:53.949684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2356d5adf1344e2b939e50ac730c4c22)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:53.952623 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:54.045933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2356d5adf1344e2b939e50ac730c4c22'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.053924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:54.060326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2356d5adf1344e2b939e50ac730c4c22'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.061573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2356d5adf1344e2b939e50ac730c4c22'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.063015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2356d5adf1344e2b939e50ac730c4c22'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.074030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-08338835-2276-4779-92e0-abb8f38655a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2356d5adf1344e2b939e50ac730c4c22'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.108734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 994/1958] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:53 2022] DELETE /identity/v3/users/2356d5adf1344e2b939e50ac730c4c22 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:54.569475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:54.580170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:54.608846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4296a-30c0-4a32-b547-3fb6e5ced87b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:54.622655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:54.674412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:54.675211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:54.675677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:54.675952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] PATH_INFO: `/v3/projects/3c2a7d47db0c4daa9420317a845ed726` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:54.677127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c2a7d47db0c4daa9420317a845ed726)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:54.679227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:54.732916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c2a7d47db0c4daa9420317a845ed726', 'request_id': 'req-7c61fda1-2581-434c-84dd-5abd6a253eb8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.743044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c2a7d47db0c4daa9420317a845ed726', 'request_id': 'req-7c61fda1-2581-434c-84dd-5abd6a253eb8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.754716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c2a7d47db0c4daa9420317a845ed726', 'request_id': 'req-7c61fda1-2581-434c-84dd-5abd6a253eb8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.774975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] The token cache is being invalidate because project 3c2a7d47db0c4daa9420317a845ed726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:31:54.775277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.776044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7c61fda1-2581-434c-84dd-5abd6a253eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:54.777818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 965/1959] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:54 2022] DELETE /identity/v3/projects/3c2a7d47db0c4daa9420317a845ed726 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:31:54.783338 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da4260-0990-454c-8690-98f626618182 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:54.783626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da4260-0990-454c-8690-98f626618182 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:54.783900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da4260-0990-454c-8690-98f626618182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:54.804339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-51da4260-0990-454c-8690-98f626618182 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:54.805739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-51da4260-0990-454c-8690-98f626618182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:54.885176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 995/1960] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:31:54.893643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d42279-3616-4b10-813d-276a15d82fab None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:54.893929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d42279-3616-4b10-813d-276a15d82fab None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:54.894205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d42279-3616-4b10-813d-276a15d82fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:54.913908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-69d42279-3616-4b10-813d-276a15d82fab None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:54.915129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-69d42279-3616-4b10-813d-276a15d82fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:54.923049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 966/1961] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:54.927393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da4260-0990-454c-8690-98f626618182 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:54.937846 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da4260-0990-454c-8690-98f626618182 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:54.942824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:54.943513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:54.943798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:54.944072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:54.944728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:31:54.945034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:54.946723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:54.959095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 996/1962] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:31:54.963699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d42279-3616-4b10-813d-276a15d82fab None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:54.973848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d42279-3616-4b10-813d-276a15d82fab None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:54.978677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:54.979302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:54.979653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:54.979928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:54.980683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:54.982614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.013100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 967/1963] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:54 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:55.017860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.028341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c6630-cf42-4c2b-b254-9db9e90c3dd5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.086509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.087113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.087412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.087680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.088305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.090287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.117428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 997/1964] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:55 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:55.118627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.130125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d42b8-2162-4d82-86ed-2b48e876b914 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.135179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.135780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.136066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.136341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.136988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.139215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.150543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 968/1965] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:55.155008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.166012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac31e76-1fd1-48e5-bea6-8c89ac2bab6f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.171187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85362804-1562-44f2-a053-e585d3201e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.171792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362804-1562-44f2-a053-e585d3201e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.172073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362804-1562-44f2-a053-e585d3201e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.172478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362804-1562-44f2-a053-e585d3201e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.173130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85362804-1562-44f2-a053-e585d3201e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.174892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85362804-1562-44f2-a053-e585d3201e1b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.186303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 998/1966] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:55.191035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.201988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed355-b152-4ade-b517-aa557e73e884 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.206711 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.207367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.207645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.207916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] PATH_INFO: `/v3/projects/99c683eeef984c70a9d7e2cb88ae727c/users/85e38a4940744829992c3d5ab1564d34/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.223247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c683eeef984c70a9d7e2cb88ae727c, user_id=85e38a4940744829992c3d5ab1564d34, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.226337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.241759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 969/1967] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:55 2022] PUT /identity/v3/projects/99c683eeef984c70a9d7e2cb88ae727c/users/85e38a4940744829992c3d5ab1564d34/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:55.253012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c61e10-785b-4098-a1ac-70306d86d971 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.253330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c61e10-785b-4098-a1ac-70306d86d971 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.253624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c61e10-785b-4098-a1ac-70306d86d971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.287245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-47c61e10-785b-4098-a1ac-70306d86d971 None None] MFA Rules not processed for user `85e38a4940744829992c3d5ab1564d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:55.288454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-47c61e10-785b-4098-a1ac-70306d86d971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:55.313836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:55.361156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 999/1968] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:55.367395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.427080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:55.449043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026761d4-8c9a-49d3-a2f0-a044ee94f415 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.454301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.454904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.455182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.455451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.456068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.458015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.484837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 970/1969] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:55.488862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c61e10-785b-4098-a1ac-70306d86d971 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.500165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c61e10-785b-4098-a1ac-70306d86d971 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.556016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.556550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.556824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.557091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.557749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.559651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.579510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1000/1970] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:55 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:55.583525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.595570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a1142-26c1-47cb-abf6-30257e11cefe admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.601523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.602264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.602670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.602952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.603601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.605205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.621124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 971/1971] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:55.621710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.633718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1a0a6-4003-45a9-b1bc-49e72fdd63dd admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.637157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920bdde-c274-41f8-83df-c7598efbf559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.637808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920bdde-c274-41f8-83df-c7598efbf559 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.638084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920bdde-c274-41f8-83df-c7598efbf559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.638353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920bdde-c274-41f8-83df-c7598efbf559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.639031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920bdde-c274-41f8-83df-c7598efbf559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.640737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920bdde-c274-41f8-83df-c7598efbf559 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.651350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1001/1972] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:31:55.656157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.677073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826d02-1565-445b-9f8b-d7d709e0241a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.682053 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.682722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.683003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.683276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] PATH_INFO: `/v3/projects/76b77c0f11904abb8fd08e6001a9e6ae/users/12e4153bb3414896bf598a21479e3d0e/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.700028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b77c0f11904abb8fd08e6001a9e6ae, user_id=12e4153bb3414896bf598a21479e3d0e, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.702808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.728236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 972/1973] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:55 2022] PUT /identity/v3/projects/76b77c0f11904abb8fd08e6001a9e6ae/users/12e4153bb3414896bf598a21479e3d0e/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:55.743528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf217-d183-4011-8348-d38bc54a7143 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.744005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf217-d183-4011-8348-d38bc54a7143 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.744389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf217-d183-4011-8348-d38bc54a7143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.787401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e6cbf217-d183-4011-8348-d38bc54a7143 None None] MFA Rules not processed for user `12e4153bb3414896bf598a21479e3d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:55.788772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e6cbf217-d183-4011-8348-d38bc54a7143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:55.874548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1002/1974] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:55.879313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.944352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69e0df-a7b1-48dc-b921-dd6a138bdb75 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:55.949037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:55.949677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:55.949960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:55.950279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:55.951425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:55.953397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:55.980800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 973/1975] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:55.984923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbf217-d183-4011-8348-d38bc54a7143 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:55.995476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbf217-d183-4011-8348-d38bc54a7143 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:56.028639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:56.058869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:56.059487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:56.059761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:56.060027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:56.060646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:56.062595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:56.088111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1003/1976] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:31:55 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:31:56.089321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:56.100751 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866abc26-7e8e-44ee-a895-98669a229627 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:56.105432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:56.106030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:56.106429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:56.106572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:56.107241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:56.108900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:56.122576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 974/1977] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:56.123826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:56.134118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471859-35e0-4979-a24b-f1fbda1f98f7 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:56.139213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:56.139880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:56.140157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:56.140598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] PATH_INFO: `/v3/projects/c64347f1a90440199e4e79949de0674b/users/b85ada5478b34dff9ecf05aa62c3cc83/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:56.156161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64347f1a90440199e4e79949de0674b, user_id=b85ada5478b34dff9ecf05aa62c3cc83, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:56.159347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:56.176024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1004/1978] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:56 2022] PUT /identity/v3/projects/c64347f1a90440199e4e79949de0674b/users/b85ada5478b34dff9ecf05aa62c3cc83/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:56.177502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:56.187558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38060b-1d17-4dde-a772-8424987e1a6d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:56.242970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eef83-7a45-4506-92ec-5e99bdd6e02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:56.243572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eef83-7a45-4506-92ec-5e99bdd6e02b admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:56.243875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eef83-7a45-4506-92ec-5e99bdd6e02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:56.244239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eef83-7a45-4506-92ec-5e99bdd6e02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:56.244910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99eef83-7a45-4506-92ec-5e99bdd6e02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:56.246538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99eef83-7a45-4506-92ec-5e99bdd6e02b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:56.255986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 975/1979] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:31:56.260130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:56.270603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff62c3-ec04-4fa1-bd0d-f05b91ff3b56 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:56.275095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:56.275731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:56.276007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:56.276279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] PATH_INFO: `/v3/projects/c64347f1a90440199e4e79949de0674b/users/b85ada5478b34dff9ecf05aa62c3cc83/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:56.278216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64347f1a90440199e4e79949de0674b, user_id=b85ada5478b34dff9ecf05aa62c3cc83, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:56.281616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:56.291335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1005/1980] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:31:56 2022] PUT /identity/v3/projects/c64347f1a90440199e4e79949de0674b/users/b85ada5478b34dff9ecf05aa62c3cc83/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:56.301589 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:56.301884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:56.302161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:56.312457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:56.335586 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] MFA Rules not processed for user `b85ada5478b34dff9ecf05aa62c3cc83`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:56.336802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:56.407306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 976/1981] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:56.418571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:56.420243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:56.481696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f7bfe-e59a-4b64-bee8-efadea497cd1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:56.485316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfc09f-2f0b-4a50-8144-c42ea79d955a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:56.485962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfc09f-2f0b-4a50-8144-c42ea79d955a service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:56.486244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfc09f-2f0b-4a50-8144-c42ea79d955a service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:56.486518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfc09f-2f0b-4a50-8144-c42ea79d955a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:56.545485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfc09f-2f0b-4a50-8144-c42ea79d955a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:56.546915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfc09f-2f0b-4a50-8144-c42ea79d955a service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:56.555700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1006/1982] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:58.728425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:58.730146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:58.814380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3e29-c5b2-4654-a928-37765ca69e58 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:58.819219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:58.820314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:58.820314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:58.820694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] PATH_INFO: `/v3/users/d1ff5b6b63da412f8981c711da67c647` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:58.822071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ff5b6b63da412f8981c711da67c647)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:58.824210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:58.886752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ff5b6b63da412f8981c711da67c647'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:58.893850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ff5b6b63da412f8981c711da67c647'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:58.901975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ff5b6b63da412f8981c711da67c647'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:58.903192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ff5b6b63da412f8981c711da67c647'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:58.903816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ff5b6b63da412f8981c711da67c647'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:31:58.925324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 977/1983] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:31:58 2022] DELETE /identity/v3/users/d1ff5b6b63da412f8981c711da67c647 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:31:58.929086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:58.929895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:58.930343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:58.958818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:31:58.961695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:59.005831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:59.042080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1007/1984] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:31:59.053119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:59.054827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:59.068165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:31:59.133059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affbe3-398d-4256-a2b5-daf7c6a55ff6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:59.137529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:59.138929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:59.139316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:59.139720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:59.150671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:59.151973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:59.152754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:59.155323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:59.171935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 978/1985] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:31:59.228488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606217-01a1-43e7-9589-0d75492637b9 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:59.241490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:59.242596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:59.242880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:59.243138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:59.245752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:59.423745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:59.425075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:31:59.448649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1008/1986] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:31:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:31:59.873360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:31:59.873360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:31:59.884383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fb52c-bff0-4d1c-97ba-84f114a2760a service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:31:59.895478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:31:59.896678 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:31:59.897004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:31:59.897491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] PATH_INFO: `/v3/projects/da37cde3ab2e49c38f99d0f5c9ba6a54` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:31:59.898730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=da37cde3ab2e49c38f99d0f5c9ba6a54)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:31:59.900811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:00.002050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da37cde3ab2e49c38f99d0f5c9ba6a54', 'request_id': 'req-7b170642-b949-49cb-9a8a-6b3e99edf98e'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.009136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da37cde3ab2e49c38f99d0f5c9ba6a54', 'request_id': 'req-7b170642-b949-49cb-9a8a-6b3e99edf98e'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.021340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da37cde3ab2e49c38f99d0f5c9ba6a54', 'request_id': 'req-7b170642-b949-49cb-9a8a-6b3e99edf98e'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.036138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] The token cache is being invalidate because project da37cde3ab2e49c38f99d0f5c9ba6a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:00.036619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.037490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.039607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 979/1987] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:31:59 2022] DELETE /identity/v3/projects/da37cde3ab2e49c38f99d0f5c9ba6a54 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:00.043397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:00.057129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d8b9-42bc-4aac-a655-4616c25a2810 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:00.149591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:00.150346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:00.150627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:00.150907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] PATH_INFO: `/v3/users/3623113d27824ec9bcda950e2da0ec01` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:00.153542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3623113d27824ec9bcda950e2da0ec01)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:00.156227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:00.226651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:00.244965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3623113d27824ec9bcda950e2da0ec01'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.259011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3623113d27824ec9bcda950e2da0ec01'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.259909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3623113d27824ec9bcda950e2da0ec01'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.260628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3623113d27824ec9bcda950e2da0ec01'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.269673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3623113d27824ec9bcda950e2da0ec01'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:00.307522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1009/1988] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:00 2022] DELETE /identity/v3/users/3623113d27824ec9bcda950e2da0ec01 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:00.600464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:00.602196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:00.675559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:00.680351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:00.681835 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b170642-b949-49cb-9a8a-6b3e99edf98e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:00.757844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:00.758607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:00.758986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:00.759269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:00.761434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:00.768956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:00.827445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d009ca8-5fb5-4d47-8208-eec22714ee56 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:00.832821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:00.833832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:00.834412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:00.835129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:00.838054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:00.885854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:00.887372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:00.909344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 980/1989] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:00.910554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:00.912028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:00.989258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:00.990950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:01.007770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea8d8-372a-4e87-a616-720ecbf631ab service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:01.016241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1010/1990] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:01.019566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:01.020227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:01.021615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:01.021901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] PATH_INFO: `/v3/projects/7b62d92a1ac94cf28fda666f6b60657e` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:01.025225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b62d92a1ac94cf28fda666f6b60657e)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:01.031714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:01.114505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b62d92a1ac94cf28fda666f6b60657e', 'request_id': 'req-5852071f-6f25-4779-98e5-e0ed46a6c571'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.124492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b62d92a1ac94cf28fda666f6b60657e', 'request_id': 'req-5852071f-6f25-4779-98e5-e0ed46a6c571'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.147538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b62d92a1ac94cf28fda666f6b60657e', 'request_id': 'req-5852071f-6f25-4779-98e5-e0ed46a6c571'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.164666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] The token cache is being invalidate because project 7b62d92a1ac94cf28fda666f6b60657e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:01.165054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.165765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.167600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 981/1991] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:00 2022] DELETE /identity/v3/projects/7b62d92a1ac94cf28fda666f6b60657e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:01.172032 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:01.187117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673fc5e-2523-4f0c-95d1-5ca7b77bcd36 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:01.262297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:01.262969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:01.263286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:01.263568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] PATH_INFO: `/v3/users/6671e6a9672340d5915879ae1914ef67` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:01.264718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6671e6a9672340d5915879ae1914ef67)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:01.266814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:01.280369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:01.330837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6671e6a9672340d5915879ae1914ef67'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.340011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6671e6a9672340d5915879ae1914ef67'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.340784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6671e6a9672340d5915879ae1914ef67'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.342985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6671e6a9672340d5915879ae1914ef67'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.353725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1f57a19e-c707-4a3d-8eb9-51adb6539f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6671e6a9672340d5915879ae1914ef67'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:01.375699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1011/1992] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:01 2022] DELETE /identity/v3/users/6671e6a9672340d5915879ae1914ef67 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:01.880007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:01.881456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:01.902916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852071f-6f25-4779-98e5-e0ed46a6c571 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:01.937867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:01.974516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:01.975019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:01.975366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:01.975729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] PATH_INFO: `/v3/projects/268e8eaa712b4e0cb78cde7ad9a3d01c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:01.977062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=268e8eaa712b4e0cb78cde7ad9a3d01c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:01.979208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.045100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268e8eaa712b4e0cb78cde7ad9a3d01c', 'request_id': 'req-6cf71039-5d00-4e24-b01c-db35cb8a69eb'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.051323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268e8eaa712b4e0cb78cde7ad9a3d01c', 'request_id': 'req-6cf71039-5d00-4e24-b01c-db35cb8a69eb'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.059209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268e8eaa712b4e0cb78cde7ad9a3d01c', 'request_id': 'req-6cf71039-5d00-4e24-b01c-db35cb8a69eb'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.070693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] The token cache is being invalidate because project 268e8eaa712b4e0cb78cde7ad9a3d01c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:02.070988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.071634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6cf71039-5d00-4e24-b01c-db35cb8a69eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.073124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 982/1993] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:01 2022] DELETE /identity/v3/projects/268e8eaa712b4e0cb78cde7ad9a3d01c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:02.079512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357d614-e637-48cf-a0f4-7f462c566aa9 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.079796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357d614-e637-48cf-a0f4-7f462c566aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.080070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357d614-e637-48cf-a0f4-7f462c566aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.099902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8357d614-e637-48cf-a0f4-7f462c566aa9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:02.101097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8357d614-e637-48cf-a0f4-7f462c566aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:02.160990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1012/1994] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:02.169544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15489633-e8fc-4c39-896d-d8969e672d32 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.169829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15489633-e8fc-4c39-896d-d8969e672d32 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.170107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15489633-e8fc-4c39-896d-d8969e672d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.190010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-15489633-e8fc-4c39-896d-d8969e672d32 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:02.191177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-15489633-e8fc-4c39-896d-d8969e672d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:02.203436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 983/1995] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:02.204498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357d614-e637-48cf-a0f4-7f462c566aa9 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.219496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357d614-e637-48cf-a0f4-7f462c566aa9 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.225094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.225855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.226142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.226411 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.227081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:02.227379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.229774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.244112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1013/1996] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:02.248929 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15489633-e8fc-4c39-896d-d8969e672d32 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.261784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15489633-e8fc-4c39-896d-d8969e672d32 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.275268 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.275969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.276409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.276757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.277756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.280600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.318510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 984/1997] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:02 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:02.323009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.352441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaf587-83ab-40dc-9380-0fb443ff5234 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.392944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.394749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:02.452340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.453002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.453302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.453835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.454478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.456751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.487236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4058-2db9-45f9-8bda-adee260cc2c0 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.488518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.489350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.489642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.489920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] PATH_INFO: `/v3/users/03b625aeaaaa41d5b77d68298fd11e84` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.491185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b625aeaaaa41d5b77d68298fd11e84)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.495567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1014/1998] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:02 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:02.499971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.502115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.508021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:02.512622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb0d9-7fea-45b8-9408-c0bfcc791e5e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.520258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.520928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.521840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.522200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.523145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.526398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.542534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1015/1999] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:02.550398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.559044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:02.563653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873176a-a86a-481f-8c16-d4b13517fc03 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.568837 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.569479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.569814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.570150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.570902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.571612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b625aeaaaa41d5b77d68298fd11e84'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.572525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.581063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b625aeaaaa41d5b77d68298fd11e84'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.583759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1016/2000] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:02.587852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.589961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b625aeaaaa41d5b77d68298fd11e84'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.590888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b625aeaaaa41d5b77d68298fd11e84'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.591650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b47a0663-e13e-452e-af19-415131ab1b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b625aeaaaa41d5b77d68298fd11e84'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:02.605385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eb0b1-1dea-4f96-9325-142265ea8c44 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.610625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf7771-2fb4-4fe1-8cc2-d29dec3ac199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.611273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf7771-2fb4-4fe1-8cc2-d29dec3ac199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.611547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf7771-2fb4-4fe1-8cc2-d29dec3ac199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.611815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf7771-2fb4-4fe1-8cc2-d29dec3ac199 admin admin] PATH_INFO: `/v3/projects/3b6e7ebc0aea4d95b07f0f42e45ed508/users/45c37970c929487281210a4c8472b260/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.614713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 985/2001] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:02 2022] DELETE /identity/v3/users/03b625aeaaaa41d5b77d68298fd11e84 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:02.619768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.620091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.620441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.628904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf7771-2fb4-4fe1-8cc2-d29dec3ac199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6e7ebc0aea4d95b07f0f42e45ed508, user_id=45c37970c929487281210a4c8472b260, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.631714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf7771-2fb4-4fe1-8cc2-d29dec3ac199 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.640899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:02.642296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:02.645943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1017/2002] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:02 2022] PUT /identity/v3/projects/3b6e7ebc0aea4d95b07f0f42e45ed508/users/45c37970c929487281210a4c8472b260/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:02.657319 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.657604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.657895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.686177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] MFA Rules not processed for user `45c37970c929487281210a4c8472b260`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:02.687402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:02.701494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 986/2003] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:02.711034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.712411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:02.752640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1018/2004] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:02.764355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:02.765849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:02.773664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c425b38-9914-49a9-a0b6-b1f05c7516ca None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.777867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.778482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.778799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.779080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.788413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.789964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.799963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 987/2005] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:02.826439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a48a-1da3-4c32-8747-60af4c7bfd9c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:02.829728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:02.830491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:02.830770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:02.831062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:02.840727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:02.842344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:02.852707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1019/2006] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:03.238227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:03.239861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:03.265031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad675c5-0715-47ee-9d97-fdc3030b090c service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:03.278227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:03.279488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:03.279760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:03.280031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] PATH_INFO: `/v3/projects/d271c89bf34a43fcacae50251155e897` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:03.281259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d271c89bf34a43fcacae50251155e897)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:03.283217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:03.343703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd271c89bf34a43fcacae50251155e897', 'request_id': 'req-2f01e7a3-7f19-4595-ad30-534db72b8302'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.345061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:03.369296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd271c89bf34a43fcacae50251155e897', 'request_id': 'req-2f01e7a3-7f19-4595-ad30-534db72b8302'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.370578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd271c89bf34a43fcacae50251155e897', 'request_id': 'req-2f01e7a3-7f19-4595-ad30-534db72b8302'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.384282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] The token cache is being invalidate because project d271c89bf34a43fcacae50251155e897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:03.385121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.385931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.387447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 988/2007] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:03 2022] DELETE /identity/v3/projects/d271c89bf34a43fcacae50251155e897 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:03.392073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:03.404541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a8deb-4af8-4e19-96cb-a8642c686aa9 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:03.460013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:03.476461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:03.477419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:03.477773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:03.478293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] PATH_INFO: `/v3/users/bf9a0e04494146f6a01f003005458506` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:03.479901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf9a0e04494146f6a01f003005458506)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:03.483774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:03.549860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf9a0e04494146f6a01f003005458506'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.562083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf9a0e04494146f6a01f003005458506'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.562972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf9a0e04494146f6a01f003005458506'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.563760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf9a0e04494146f6a01f003005458506'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.575010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a0c0ba53-28e8-4e15-bbb6-978c823dcdd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf9a0e04494146f6a01f003005458506'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.608731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1020/2008] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:03 2022] DELETE /identity/v3/users/bf9a0e04494146f6a01f003005458506 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:03.704133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:03.705796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:03.729845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e7a3-7f19-4595-ad30-534db72b8302 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:03.832522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:03.833345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:03.833557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:03.833833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] PATH_INFO: `/v3/users/d02183bbf039429395a69fe0259a34cc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:03.835150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02183bbf039429395a69fe0259a34cc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:03.837385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:03.936943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02183bbf039429395a69fe0259a34cc'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.945610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02183bbf039429395a69fe0259a34cc'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.958742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02183bbf039429395a69fe0259a34cc'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.959841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02183bbf039429395a69fe0259a34cc'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.963429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02183bbf039429395a69fe0259a34cc'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:03.985438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 989/2009] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:03 2022] DELETE /identity/v3/users/d02183bbf039429395a69fe0259a34cc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:03.990955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:03.991275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:03.991614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.020952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.022683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:04.033762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:04.056876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:04.122391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:04.127391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983bf4c-b653-408f-a839-15ec386915dc admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.129718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1021/2010] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:04.137997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.140091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.140091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.140091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] PATH_INFO: `/v3/projects/0a0c13bef6e84147833f79d13fc259ee` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.140382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.141341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0c13bef6e84147833f79d13fc259ee)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.141529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:04.143682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.209450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227a512-a1fc-4bb3-88cd-d8662dab3129 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.219284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0c13bef6e84147833f79d13fc259ee', 'request_id': 'req-c5d39d24-2233-4866-80f0-30992c94a7c5'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:04.230760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0c13bef6e84147833f79d13fc259ee', 'request_id': 'req-c5d39d24-2233-4866-80f0-30992c94a7c5'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:04.245448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0c13bef6e84147833f79d13fc259ee', 'request_id': 'req-c5d39d24-2233-4866-80f0-30992c94a7c5'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:04.266933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] The token cache is being invalidate because project 0a0c13bef6e84147833f79d13fc259ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:04.267239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:04.267852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:04.269502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5d39d24-2233-4866-80f0-30992c94a7c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:04.271787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 990/2011] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:04 2022] DELETE /identity/v3/projects/0a0c13bef6e84147833f79d13fc259ee => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:04.278471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc160f4-5c4a-49a2-9cb6-a335f1dd153a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.279227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc160f4-5c4a-49a2-9cb6-a335f1dd153a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.279717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc160f4-5c4a-49a2-9cb6-a335f1dd153a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.305293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2fc160f4-5c4a-49a2-9cb6-a335f1dd153a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:04.306917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2fc160f4-5c4a-49a2-9cb6-a335f1dd153a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:04.328322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.329061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.329418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.329753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.332363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:04.401973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 991/2012] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:04.422920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd6f8-5311-4c26-89ad-108686e87ef3 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.423368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd6f8-5311-4c26-89ad-108686e87ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.423751 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd6f8-5311-4c26-89ad-108686e87ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.434321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.436476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.449695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-37cdd6f8-5311-4c26-89ad-108686e87ef3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:04.450052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1022/2013] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:04.451450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-37cdd6f8-5311-4c26-89ad-108686e87ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:04.463470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 992/2014] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:04.467637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.479376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeeedf2-8276-4740-8a6f-f7a581b4c320 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.485210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.485938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.486191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.486415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.487158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:04.487404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.488993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.504008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1023/2015] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:32:04.508863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdd6f8-5311-4c26-89ad-108686e87ef3 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.518608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdd6f8-5311-4c26-89ad-108686e87ef3 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.523614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.524315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.524701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.525043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.525766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.528326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.563618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 993/2016] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:04 2022] POST /identity/v3/projects => generated 407 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:04.567743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.582295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9335d-695d-4e04-a2af-f377ce89f0f1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.678181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.678898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.679181 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.679997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.680937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.683182 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.711917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1024/2017] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:04 2022] POST /identity/v3/users => generated 304 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:04.715722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.736507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a994c-0518-462b-844c-baf988cecfe5 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.742599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.743303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.744091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.744441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.745224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.747527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.760808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 994/2018] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:04.765751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.790886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbe812-4484-4cf7-b342-b5fb8a8d52d0 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.814056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.814680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.814963 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.815235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.815887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.817665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.837100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1025/2019] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:04.838991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.860472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192ee4-23a8-4ba5-baa4-5a1c18247e4d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.869242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f0f88-00ed-4b91-921a-671ddcaaf4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.873072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f0f88-00ed-4b91-921a-671ddcaaf4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.874345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f0f88-00ed-4b91-921a-671ddcaaf4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.875056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f0f88-00ed-4b91-921a-671ddcaaf4c2 admin admin] PATH_INFO: `/v3/projects/9532ceafd6ab4f418964f0599804beaa/users/cf690560b43c488ba7921945ce1faf41/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.894848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:04.896993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:04.915398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c584-8a58-42b1-96a0-a44089c1cb1a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:04.920002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f0f88-00ed-4b91-921a-671ddcaaf4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9532ceafd6ab4f418964f0599804beaa, user_id=cf690560b43c488ba7921945ce1faf41, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.923664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f0f88-00ed-4b91-921a-671ddcaaf4c2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.944137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:04.944821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.945094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.945349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] PATH_INFO: `/v3/projects/e100286925174f36b3260211cb564c5c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:04.947956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e100286925174f36b3260211cb564c5c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:04.950136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:04.970299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 995/2020] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:04 2022] PUT /identity/v3/projects/9532ceafd6ab4f418964f0599804beaa/users/cf690560b43c488ba7921945ce1faf41/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:04.975876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:04.976171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:04.976435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.019808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] MFA Rules not processed for user `cf690560b43c488ba7921945ce1faf41`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:05.021434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.024148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:05.037748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:05.061509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e100286925174f36b3260211cb564c5c', 'request_id': 'req-b428f45e-7873-45c6-8364-4c6fc0636a25'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.070506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e100286925174f36b3260211cb564c5c', 'request_id': 'req-b428f45e-7873-45c6-8364-4c6fc0636a25'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.082055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e100286925174f36b3260211cb564c5c', 'request_id': 'req-b428f45e-7873-45c6-8364-4c6fc0636a25'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.103777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] The token cache is being invalidate because project e100286925174f36b3260211cb564c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:05.104287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.106645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.109141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1026/2021] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:04 2022] DELETE /identity/v3/projects/e100286925174f36b3260211cb564c5c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:05.113507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.115254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.135028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 996/2022] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:05.139353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.140793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.237067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f45e-7873-45c6-8364-4c6fc0636a25 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:05.244677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:05.245415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:05.245795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:05.246083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] PATH_INFO: `/v3/users/4f7dbdc2447143a4958e6376f466f566` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.247370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7dbdc2447143a4958e6376f466f566)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:05.249021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df1224-94d1-4e65-949d-c9c1980505e8 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:05.250223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:05.256921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:05.257718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:05.259210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:05.259660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.260634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:05.262770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:05.335237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 997/2023] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:05 2022] POST /identity/v3/projects => generated 409 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:05.336529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.360031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7dbdc2447143a4958e6376f466f566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.374650 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7dbdc2447143a4958e6376f466f566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.375946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7dbdc2447143a4958e6376f466f566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.376994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7dbdc2447143a4958e6376f466f566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.392794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7dbdc2447143a4958e6376f466f566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.442352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1027/2024] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:05 2022] DELETE /identity/v3/users/4f7dbdc2447143a4958e6376f466f566 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:05.443096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.444424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.473505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec27bf1-1b7a-41fa-9f1c-d8647ee76c8d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:05.501543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:05.530557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe7b3-6093-4dc4-88bf-52f71135debd admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:05.535560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:05.536205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:05.536823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:05.537105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] PATH_INFO: `/v3/users/85e38a4940744829992c3d5ab1564d34` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.538557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e38a4940744829992c3d5ab1564d34)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:05.539501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:05.541325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:05.561124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:05.562035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:05.562318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:05.562587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.563227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:05.565727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:05.614836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 998/2025] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:05 2022] POST /identity/v3/users => generated 311 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:05.619512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.622004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.639920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbaf57-4c60-446c-95bc-3c6c328cb174 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:05.643842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e38a4940744829992c3d5ab1564d34'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.659459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e38a4940744829992c3d5ab1564d34'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.660777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e38a4940744829992c3d5ab1564d34'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.662012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e38a4940744829992c3d5ab1564d34'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.667907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:05.671510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:05.672031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:05.673313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.674038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:05.679228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-989aa547-4241-4a5f-9f02-54df105ab6a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e38a4940744829992c3d5ab1564d34'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:05.681497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:05.708963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 999/2026] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:05.715265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.717679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.731027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1028/2027] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:05 2022] DELETE /identity/v3/users/85e38a4940744829992c3d5ab1564d34 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:05.736312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:05.736597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:05.736990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.753313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a6b9-afa8-426d-acfa-7bfc7c9736d3 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:05.776797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:05.778219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.879527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1029/2028] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:05.883376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:05.884155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:05.884349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:05.884697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] PATH_INFO: `/v3/projects/71e3d271ed0a47b28c56d9db3660923e/users/df3514ac980344e2b05978eaef1c2c60/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:05.903352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e3d271ed0a47b28c56d9db3660923e, user_id=df3514ac980344e2b05978eaef1c2c60, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:05.906748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:05.914070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.915461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:05.928152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1000/2029] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:05 2022] PUT /identity/v3/projects/71e3d271ed0a47b28c56d9db3660923e/users/df3514ac980344e2b05978eaef1c2c60/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:05.932441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:05.997175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bb221-beae-4755-a471-a6d7becc2f58 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.022664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:06.059783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94618bd-8072-4275-bdaf-59369502f77e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.068293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.068877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.069160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.069465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.070275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.071900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:06.073463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab2e49-76de-4173-978f-f70c2d7e2c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.074116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2e49-76de-4173-978f-f70c2d7e2c74 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.074400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2e49-76de-4173-978f-f70c2d7e2c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.074674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2e49-76de-4173-978f-f70c2d7e2c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.083242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1001/2030] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:06.084591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab2e49-76de-4173-978f-f70c2d7e2c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.086104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab2e49-76de-4173-978f-f70c2d7e2c74 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:06.087368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:06.101009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1030/2031] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:06.102807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ada79-f523-4b0f-9287-9cb8178de3bc admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.108789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.109508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.109676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.109947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] PATH_INFO: `/v3/projects/71e3d271ed0a47b28c56d9db3660923e/users/df3514ac980344e2b05978eaef1c2c60/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.112086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e3d271ed0a47b28c56d9db3660923e, user_id=df3514ac980344e2b05978eaef1c2c60, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.114863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:06.127189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1002/2032] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:06 2022] PUT /identity/v3/projects/71e3d271ed0a47b28c56d9db3660923e/users/df3514ac980344e2b05978eaef1c2c60/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:06.138001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.138391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.138761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.172744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] MFA Rules not processed for user `df3514ac980344e2b05978eaef1c2c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:06.175186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:06.263390 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1031/2033] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:06.276867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:06.281529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:06.330963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:06.376272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976bba2-3a23-4637-9818-d4261823bc68 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.380125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.380839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.381110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.381507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.391213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.392744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:06.403562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1003/2034] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:06 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:06.443503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:06.445018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:06.540713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb2b4d-cf59-450b-bf81-0cc19a33e036 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.547535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.548620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.548620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.548782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] PATH_INFO: `/v3/users/45c37970c929487281210a4c8472b260` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.550785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c37970c929487281210a4c8472b260)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.553259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:06.639797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c37970c929487281210a4c8472b260'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:06.647904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:06.669254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c37970c929487281210a4c8472b260'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:06.669981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c37970c929487281210a4c8472b260'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:06.670861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c37970c929487281210a4c8472b260'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:06.676542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:06.681871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c37970c929487281210a4c8472b260'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:06.688104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418ce52-77a5-4318-948e-e3a474f5becc service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.695465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe620e-1eef-4365-a7bf-5cab62ace24f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.696082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe620e-1eef-4365-a7bf-5cab62ace24f service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.696358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe620e-1eef-4365-a7bf-5cab62ace24f service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.696627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe620e-1eef-4365-a7bf-5cab62ace24f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.698411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f5fe620e-1eef-4365-a7bf-5cab62ace24f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:06.742283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1032/2035] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:06 2022] DELETE /identity/v3/users/45c37970c929487281210a4c8472b260 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:06.745996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:06.753752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:06.827665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe620e-1eef-4365-a7bf-5cab62ace24f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.834233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe620e-1eef-4365-a7bf-5cab62ace24f service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:06.864514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1004/2036] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:06 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:06.867679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.867969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.869549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.912910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8cab6-b48c-4a70-8c52-46e1dd6f0577 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.917081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:06.918440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:06.927168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.928423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1005/2037] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:06.929594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.930346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.931525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] PATH_INFO: `/v3/projects/aee1b86c998f4e67870f0311913b9a91` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.937688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee1b86c998f4e67870f0311913b9a91)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.942658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:06.950170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:06.957025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:06.977251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2613f-3c92-45a2-a077-5a1c29a6d080 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:06.985188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:06.987049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:06.987619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:06.987993 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] PATH_INFO: `/v3/projects/99c683eeef984c70a9d7e2cb88ae727c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:06.990045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c683eeef984c70a9d7e2cb88ae727c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:06.992258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:07.033017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee1b86c998f4e67870f0311913b9a91', 'request_id': 'req-3a918642-42fd-4497-9c47-2d191b733a36'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.043042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee1b86c998f4e67870f0311913b9a91', 'request_id': 'req-3a918642-42fd-4497-9c47-2d191b733a36'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.059298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee1b86c998f4e67870f0311913b9a91', 'request_id': 'req-3a918642-42fd-4497-9c47-2d191b733a36'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.063971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c683eeef984c70a9d7e2cb88ae727c', 'request_id': 'req-2cc1648a-172e-4418-9213-431e67f5616c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.077777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c683eeef984c70a9d7e2cb88ae727c', 'request_id': 'req-2cc1648a-172e-4418-9213-431e67f5616c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.086209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] The token cache is being invalidate because project aee1b86c998f4e67870f0311913b9a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:07.086740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.087738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.089750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c683eeef984c70a9d7e2cb88ae727c', 'request_id': 'req-2cc1648a-172e-4418-9213-431e67f5616c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.097099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1033/2038] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:06 2022] DELETE /identity/v3/projects/aee1b86c998f4e67870f0311913b9a91 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:07.097099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:07.099781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:07.122052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] The token cache is being invalidate because project 99c683eeef984c70a9d7e2cb88ae727c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:07.122052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.123150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.128247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1006/2039] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:06 2022] DELETE /identity/v3/projects/99c683eeef984c70a9d7e2cb88ae727c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:07.141461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:07.141461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:07.238229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a918642-42fd-4497-9c47-2d191b733a36 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:07.249483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:07.251932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:07.300140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1648a-172e-4418-9213-431e67f5616c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:07.311526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:07.312383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:07.312666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:07.313056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] PATH_INFO: `/v3/users/8609691780d2420ca5228c4e154736fd` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:07.314708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8609691780d2420ca5228c4e154736fd)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:07.323230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:07.323967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:07.324531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:07.325006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:07.325655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:07.341416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:07.365946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:07.367866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:07.382974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1034/2040] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:07.383987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:07.385534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:07.399744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55ddd9-d535-48d4-acad-ab64de186552 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:07.409299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:07.409954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:07.410233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:07.410506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] PATH_INFO: `/v3/users/12e4153bb3414896bf598a21479e3d0e` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:07.414675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e4153bb3414896bf598a21479e3d0e)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:07.421351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:07.496379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8609691780d2420ca5228c4e154736fd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.515560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8609691780d2420ca5228c4e154736fd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.536150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e4153bb3414896bf598a21479e3d0e'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.542043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8609691780d2420ca5228c4e154736fd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.542630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8609691780d2420ca5228c4e154736fd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.543151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8609691780d2420ca5228c4e154736fd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.548773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e4153bb3414896bf598a21479e3d0e'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.549475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e4153bb3414896bf598a21479e3d0e'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.550195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e4153bb3414896bf598a21479e3d0e'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.567695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ade3b30d-23d9-49b4-929b-c60f76e18d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e4153bb3414896bf598a21479e3d0e'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:07.586339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1007/2041] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:07 2022] DELETE /identity/v3/users/8609691780d2420ca5228c4e154736fd => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:07.614679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1035/2042] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:07 2022] DELETE /identity/v3/users/12e4153bb3414896bf598a21479e3d0e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:08.272874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:08.274465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:08.425602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d870e-2e93-4d9b-b587-ac3a2664392e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:08.497242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:08.498700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:08.499052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:08.499442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] PATH_INFO: `/v3/projects/3b6e7ebc0aea4d95b07f0f42e45ed508` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:08.500756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6e7ebc0aea4d95b07f0f42e45ed508)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:08.502956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:08.523435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:08.585474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6e7ebc0aea4d95b07f0f42e45ed508', 'request_id': 'req-ca2643c9-a121-4ce6-869a-b6b11fd67be7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.585474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6e7ebc0aea4d95b07f0f42e45ed508', 'request_id': 'req-ca2643c9-a121-4ce6-869a-b6b11fd67be7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.598310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6e7ebc0aea4d95b07f0f42e45ed508', 'request_id': 'req-ca2643c9-a121-4ce6-869a-b6b11fd67be7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.617037 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] The token cache is being invalidate because project 3b6e7ebc0aea4d95b07f0f42e45ed508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:08.617525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.618357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.626887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1008/2043] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:08 2022] DELETE /identity/v3/projects/3b6e7ebc0aea4d95b07f0f42e45ed508 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:08.630286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:08.630808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:08.630808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:08.679544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:08.680877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:08.691061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:08.705552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:08.806251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1036/2044] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:08.807472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:08.809047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:08.844415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44951da1-bd98-49d2-ac9d-c57398d8a3d8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:08.849047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2643c9-a121-4ce6-869a-b6b11fd67be7 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:08.860206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:08.862893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:08.863323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:08.863938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] PATH_INFO: `/v3/projects/76b77c0f11904abb8fd08e6001a9e6ae` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:08.864109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:08.864916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:08.865165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:08.865486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] PATH_INFO: `/v3/projects/dae150012d1146fd92212b8a9d72e15f` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:08.868494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae150012d1146fd92212b8a9d72e15f)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:08.868626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b77c0f11904abb8fd08e6001a9e6ae)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:08.871152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:08.871617 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:08.926527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:08.973877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae150012d1146fd92212b8a9d72e15f', 'request_id': 'req-b9059449-f4ba-4b43-8821-55ec57373255'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.977948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b77c0f11904abb8fd08e6001a9e6ae', 'request_id': 'req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.990002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae150012d1146fd92212b8a9d72e15f', 'request_id': 'req-b9059449-f4ba-4b43-8821-55ec57373255'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:08.999135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b77c0f11904abb8fd08e6001a9e6ae', 'request_id': 'req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.004756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae150012d1146fd92212b8a9d72e15f', 'request_id': 'req-b9059449-f4ba-4b43-8821-55ec57373255'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.014267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b77c0f11904abb8fd08e6001a9e6ae', 'request_id': 'req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.025544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] The token cache is being invalidate because project dae150012d1146fd92212b8a9d72e15f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:09.025845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.037776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b9059449-f4ba-4b43-8821-55ec57373255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.040178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1009/2045] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:08 2022] DELETE /identity/v3/projects/dae150012d1146fd92212b8a9d72e15f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:09.042732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] The token cache is being invalidate because project 76b77c0f11904abb8fd08e6001a9e6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:09.043036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.043780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3a6ff5c4-35df-4e1d-a8f4-fe4944cddfe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.045901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817b936-988d-4bb0-bb98-a6faa5748502 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.046206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1037/2046] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:08 2022] DELETE /identity/v3/projects/76b77c0f11904abb8fd08e6001a9e6ae => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:09.047460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.047743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.048015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.048877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817b936-988d-4bb0-bb98-a6faa5748502 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.049180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817b936-988d-4bb0-bb98-a6faa5748502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.075386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:09.075609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4817b936-988d-4bb0-bb98-a6faa5748502 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:09.077032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4817b936-988d-4bb0-bb98-a6faa5748502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:09.077573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:09.125671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:09.168541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1038/2047] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:09.170448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.173820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1010/2048] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:09.175591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817b936-988d-4bb0-bb98-a6faa5748502 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.178863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:09.186980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817b936-988d-4bb0-bb98-a6faa5748502 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.192682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56ffee-d79e-4c42-9ccd-75732d107e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.193495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ffee-d79e-4c42-9ccd-75732d107e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.193991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ffee-d79e-4c42-9ccd-75732d107e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.194474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ffee-d79e-4c42-9ccd-75732d107e2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.195416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56ffee-d79e-4c42-9ccd-75732d107e2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:09.195951 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56ffee-d79e-4c42-9ccd-75732d107e2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.197968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56ffee-d79e-4c42-9ccd-75732d107e2f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.206763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91646621-2b45-4244-bf31-7aabc27c5d7f None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.217934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.218584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.218864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.219137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] PATH_INFO: `/v3/users/b85ada5478b34dff9ecf05aa62c3cc83` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.220315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85ada5478b34dff9ecf05aa62c3cc83)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.223462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.224782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1011/2049] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:09.226554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90eaa6-3099-44f0-8cef-1a1bacd37155 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.227328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90eaa6-3099-44f0-8cef-1a1bacd37155 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.227687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90eaa6-3099-44f0-8cef-1a1bacd37155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.253964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-eb90eaa6-3099-44f0-8cef-1a1bacd37155 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:09.255680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-eb90eaa6-3099-44f0-8cef-1a1bacd37155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:09.282104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1012/2050] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:09.283994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90eaa6-3099-44f0-8cef-1a1bacd37155 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.300324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85ada5478b34dff9ecf05aa62c3cc83'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.309876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90eaa6-3099-44f0-8cef-1a1bacd37155 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.311613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85ada5478b34dff9ecf05aa62c3cc83'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.313061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85ada5478b34dff9ecf05aa62c3cc83'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.314147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85ada5478b34dff9ecf05aa62c3cc83'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.321628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.323277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.324348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.324479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.325090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.326599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85ada5478b34dff9ecf05aa62c3cc83'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.330542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.364909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1039/2051] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:09 2022] DELETE /identity/v3/users/b85ada5478b34dff9ecf05aa62c3cc83 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:09.367152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.368933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:09.393590 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1013/2052] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:09.396817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.398665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:09.430164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:09.502991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b2de-1526-4320-8470-a2cfe13e7b64 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.511411 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.512418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.512878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.513895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.514985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:09.515434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.517873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.522821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23664f7-b412-4134-810d-a0c8b3b6df5b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.538282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.538981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.539259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.539548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.541324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.545621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1040/2053] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:09.545778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.547965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.568046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb34f4c-9c3a-40d1-b96d-fdac647261c1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.588514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.589447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.589738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.590017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.590668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.604269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.607615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1014/2054] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:09.614148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.639588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb32f5-82a3-4809-be20-85af29bcaaf8 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.646836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.647793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.648244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.648700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.649544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.650508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1041/2055] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:09.652286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.656293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.668986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1015/2056] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:09.677971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.694044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21634e9d-74b7-4315-9520-7f8bdb67a1f1 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.771551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cf7a4-ffb3-449d-b5d1-2610e0f86a14 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.776540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.777300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.777792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.777792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.778345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.781252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.791837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.791837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.791837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.791837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.792576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.794473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.808652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1042/2057] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/users => generated 301 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:09.809574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.812339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:09.814865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1016/2058] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:09.815778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.826259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d108cda-7042-44b1-b1fe-d0a187d43afd admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.829453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201be5b5-2a4d-4ee7-b9d9-29b687fc8211 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.831509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.832170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.832504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.832723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] PATH_INFO: `/v3/projects/c64347f1a90440199e4e79949de0674b` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.833956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64347f1a90440199e4e79949de0674b)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.834482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.835087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.835363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.835709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.835975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.836353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.838129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.841444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:09.858316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1017/2059] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:09.858316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.867653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31599e-5407-43ba-a094-fc3a942b2c2d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:09.875620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:09.876560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.877640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.877935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] PATH_INFO: `/v3/projects/197f506933244ff1a5d47d8e84255d0e/users/61038258e95144a8bc1a424192c3dbc1/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.900083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197f506933244ff1a5d47d8e84255d0e, user_id=61038258e95144a8bc1a424192c3dbc1, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:09.903049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:09.914856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64347f1a90440199e4e79949de0674b', 'request_id': 'req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.921838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64347f1a90440199e4e79949de0674b', 'request_id': 'req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.925387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1018/2060] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:09 2022] PUT /identity/v3/projects/197f506933244ff1a5d47d8e84255d0e/users/61038258e95144a8bc1a424192c3dbc1/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:09.926482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:09.940856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64347f1a90440199e4e79949de0674b', 'request_id': 'req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.962017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] The token cache is being invalidate because project c64347f1a90440199e4e79949de0674b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:09.962538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.963271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-9b2d1ac8-2af8-42ab-b70c-4eac50737fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:09.968184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1043/2061] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:09 2022] DELETE /identity/v3/projects/c64347f1a90440199e4e79949de0674b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:09.968184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1fcc6-bb65-440c-8b3f-7a21df6f4796 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:09.968184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1fcc6-bb65-440c-8b3f-7a21df6f4796 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:09.968184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1fcc6-bb65-440c-8b3f-7a21df6f4796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:09.992978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:10.013807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-11b1fcc6-bb65-440c-8b3f-7a21df6f4796 None None] MFA Rules not processed for user `61038258e95144a8bc1a424192c3dbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:10.015276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-11b1fcc6-bb65-440c-8b3f-7a21df6f4796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.035174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02102a9d-191c-42b1-8be2-a54d84d2d5d8 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.041476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf70372-1d26-4a89-86e5-5cf8282518f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.042185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70372-1d26-4a89-86e5-5cf8282518f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.042766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70372-1d26-4a89-86e5-5cf8282518f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.043046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70372-1d26-4a89-86e5-5cf8282518f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.043705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf70372-1d26-4a89-86e5-5cf8282518f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.045455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf70372-1d26-4a89-86e5-5cf8282518f4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.056925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1019/2062] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:10.058434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.058717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.058989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.081431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:10.082640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.091558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1020/2063] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:10.092374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.093680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.101533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1044/2064] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:10.104606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.105303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.105677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.106413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f617f8-04ff-44eb-b1d1-252fc22cd2be None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.111330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.111935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.112211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.112485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] PATH_INFO: `/v3/projects/b4cd99269e0f494ea94269f9c7d53ac5/users/34403fa224b04c13868d64b63231ad7c/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.127630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:10.128968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.129147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4cd99269e0f494ea94269f9c7d53ac5, user_id=34403fa224b04c13868d64b63231ad7c, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.132085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.138887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1045/2065] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:10.140194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.141738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.150383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1021/2066] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:10 2022] PUT /identity/v3/projects/b4cd99269e0f494ea94269f9c7d53ac5/users/34403fa224b04c13868d64b63231ad7c/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:10.152023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.165401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b85b5f-b86f-4091-9ca3-afaafb4adefc None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.224769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f8100-4a23-471e-bd86-ddc7f090977b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.229734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981be29-34f3-4da7-a6ea-82eb1c9382bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.230365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981be29-34f3-4da7-a6ea-82eb1c9382bf admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.230575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.230706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981be29-34f3-4da7-a6ea-82eb1c9382bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.230901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981be29-34f3-4da7-a6ea-82eb1c9382bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.231197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.231477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.231661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981be29-34f3-4da7-a6ea-82eb1c9382bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:10.232305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.232489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981be29-34f3-4da7-a6ea-82eb1c9382bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.232952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.234112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981be29-34f3-4da7-a6ea-82eb1c9382bf admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.234923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.246803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1022/2067] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:10.249088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8735e4-1739-4665-95aa-10f13bb5ff37 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.249418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8735e4-1739-4665-95aa-10f13bb5ff37 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.249693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8735e4-1739-4665-95aa-10f13bb5ff37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.266029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1046/2068] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:10.266029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.280566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-db8735e4-1739-4665-95aa-10f13bb5ff37 None None] MFA Rules not processed for user `34403fa224b04c13868d64b63231ad7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:10.282094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-db8735e4-1739-4665-95aa-10f13bb5ff37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.303801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:10.331511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a86cac-5c19-4b70-96b3-a607631c8ef9 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.336186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.336789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.337071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.337386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.338021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.339936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.354142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1023/2069] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:10.359358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8735e4-1739-4665-95aa-10f13bb5ff37 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.371328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1047/2070] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:10.371328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.371699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8735e4-1739-4665-95aa-10f13bb5ff37 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.374727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:10.427064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.428823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.429106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.429423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.430055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.431973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.432301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a611f-a279-4dd7-baf5-d55f8151ad26 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.437710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.438304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.438579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.438848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.439460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.441359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.453461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1024/2071] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:10.454254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.463737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daad12-81ea-4802-8d05-271870055e25 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.469771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.473292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1048/2072] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:10.473454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.473454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.473454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.473707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.473799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.473937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.495642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1025/2073] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/users => generated 285 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:10.500030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.509454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d45ff-2119-4b0f-8c84-9774bc69a95c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.534749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513480e6-622b-4dac-9f1a-c99230057c54 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.539450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.540058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.540339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.540611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.541253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.542885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.553064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1049/2074] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:10.553901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.563320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f9577-4420-468a-b661-3f3a2afca0c4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.565864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.566476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.566754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.567024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.567641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.567961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.568532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.568814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.569089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.569578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.569747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.571304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.588467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1050/2075] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:10.589419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.595414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1026/2076] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:10.596300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.604127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ab9a7-fc6b-46f5-9c64-390293eaae2c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.608207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd35cfc-84f3-4892-a0aa-857eb962d0a5 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.609420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.610019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.610297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.610570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.611205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.612829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.612941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.613515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.613799 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.614076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.614718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.616303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.623155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1051/2077] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:10.623964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.629301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1027/2078] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:10.630248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.633822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92428a93-115c-45f1-8e2e-3f547c821efb admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.638329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.638905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.639177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.639446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.640043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d5953-0c59-4e10-9e0d-8ca5cf5d2cbd admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.640155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.641663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.644738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.645391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.645676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.645954 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] PATH_INFO: `/v3/projects/597c02e5db224da1b11e09136b11ea2b/users/7bf3b6da1fa8416e9ff66490983808bd/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.651588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1052/2079] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:10.652551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.658437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:10.663393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=597c02e5db224da1b11e09136b11ea2b, user_id=7bf3b6da1fa8416e9ff66490983808bd, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.665312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb64e8-4f01-4a70-a454-7b7d97570780 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.667456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.670764 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3e916-47a1-4916-a931-75ac81ee2c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.671402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3e916-47a1-4916-a931-75ac81ee2c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.671679 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3e916-47a1-4916-a931-75ac81ee2c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.671948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3e916-47a1-4916-a931-75ac81ee2c0f admin admin] PATH_INFO: `/v3/projects/27cb93320e3f4e5ba64ce9541c405336/users/563fb73ed5ef408db830abadb69ebc27/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.681607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1028/2080] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:10 2022] PUT /identity/v3/projects/597c02e5db224da1b11e09136b11ea2b/users/7bf3b6da1fa8416e9ff66490983808bd/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:10.682380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.687507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3e916-47a1-4916-a931-75ac81ee2c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cb93320e3f4e5ba64ce9541c405336, user_id=563fb73ed5ef408db830abadb69ebc27, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.690199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3e916-47a1-4916-a931-75ac81ee2c0f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.702915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1053/2081] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:10 2022] PUT /identity/v3/projects/27cb93320e3f4e5ba64ce9541c405336/users/563fb73ed5ef408db830abadb69ebc27/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:10.704113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeac97-0b30-4b1e-898d-9e33ea202af4 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.704395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeac97-0b30-4b1e-898d-9e33ea202af4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.704664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeac97-0b30-4b1e-898d-9e33ea202af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.733575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-beeeac97-0b30-4b1e-898d-9e33ea202af4 None None] MFA Rules not processed for user `7bf3b6da1fa8416e9ff66490983808bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:10.734814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-beeeac97-0b30-4b1e-898d-9e33ea202af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.749718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:10.755175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc46b2-0f26-46ba-a2fd-e6b4dc281eab admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.808692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1054/2082] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:10.810107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24e2a-71d8-442c-b957-134068aec122 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.810453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24e2a-71d8-442c-b957-134068aec122 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.810793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24e2a-71d8-442c-b957-134068aec122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.822121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.822780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.823068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.823348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] PATH_INFO: `/v3/projects/3510c833e83649228a5df27d65e7228c/users/94dfb82a4e864d6f8d1526696f940f06/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.837648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3510c833e83649228a5df27d65e7228c, user_id=94dfb82a4e864d6f8d1526696f940f06, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.837790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-aee24e2a-71d8-442c-b957-134068aec122 None None] MFA Rules not processed for user `563fb73ed5ef408db830abadb69ebc27`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:10.839106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-aee24e2a-71d8-442c-b957-134068aec122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.840323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.855432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1029/2083] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:10 2022] PUT /identity/v3/projects/3510c833e83649228a5df27d65e7228c/users/94dfb82a4e864d6f8d1526696f940f06/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:10.856209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.906929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1055/2084] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:10.907766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24e2a-71d8-442c-b957-134068aec122 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.918854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24e2a-71d8-442c-b957-134068aec122 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.924767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.925394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.925613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65663a-bf66-40c5-91cd-a4ae3801a6d3 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:10.925723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.925930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.926571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.928159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.930350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:10.930944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:10.931227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:10.931508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:10.932127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:10.934161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:10.938092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1056/2085] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:10.938907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.940722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:10.963611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1030/2086] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:10 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:10.964416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:10.973325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:11.029839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7325b-c03e-4b6a-80dd-6f1bcc53dfdc admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.039186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4275cf-0b6c-4e15-80a7-3889994e1678 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.045130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.046152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.046429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.046700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] PATH_INFO: `/v3/projects/3510c833e83649228a5df27d65e7228c/users/94dfb82a4e864d6f8d1526696f940f06/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.051486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3510c833e83649228a5df27d65e7228c, user_id=94dfb82a4e864d6f8d1526696f940f06, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.054405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.073578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1031/2087] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:10 2022] PUT /identity/v3/projects/3510c833e83649228a5df27d65e7228c/users/94dfb82a4e864d6f8d1526696f940f06/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:11.074464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.078632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:11.104493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9be8c8c5f7f6417e9ab19ef5bce0f060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.105202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.105571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.105932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.151840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d6e35-8744-4656-8a2c-05475bc2c90c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.157231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24db2-4220-452d-9552-b09df4c259c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.157810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24db2-4220-452d-9552-b09df4c259c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.158126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24db2-4220-452d-9552-b09df4c259c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.158359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24db2-4220-452d-9552-b09df4c259c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.158987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24db2-4220-452d-9552-b09df4c259c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.160907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24db2-4220-452d-9552-b09df4c259c2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.172043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.173713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.183149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1057/2089] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:11.183312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1032/2089] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:11 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:11.184397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.184680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.185007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.186958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.197546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca045a-9f84-462d-bfea-59cbe72481a3 service glance] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.202918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.203516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.203799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.209581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.210337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.212200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.213437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] MFA Rules not processed for user `94dfb82a4e864d6f8d1526696f940f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:11.214802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:11.224368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1058/2090] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:11.228148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.238473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcd232-bcc0-427e-afed-db6e7d27cb3d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.243135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.243877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.244065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.244323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] PATH_INFO: `/v3/projects/54467f07001e4ac580e96f56777960e7/users/fb8e1abba297435d8a99fe8bf007597d/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.263475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54467f07001e4ac580e96f56777960e7, user_id=fb8e1abba297435d8a99fe8bf007597d, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.267342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.284068 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1059/2091] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:11 2022] PUT /identity/v3/projects/54467f07001e4ac580e96f56777960e7/users/fb8e1abba297435d8a99fe8bf007597d/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:11.287664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.290487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1033/2092] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:11.303174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.304551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:11.320436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:11.360130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507b9f-8019-4fe0-93e3-6ff686ac4f71 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.365306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.367659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.368051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c52cc-81ae-4773-abfa-b8ef869ac2fd None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.368223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.368530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.369674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.371397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.387128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1060/2093] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:11.389127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.404217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbfba6-33a2-4059-9421-456c1238a224 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.410001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1249a25-9730-4a49-98b7-7c496de2478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.410753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1249a25-9730-4a49-98b7-7c496de2478e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.411103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1249a25-9730-4a49-98b7-7c496de2478e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.411493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1249a25-9730-4a49-98b7-7c496de2478e admin admin] PATH_INFO: `/v3/projects/54467f07001e4ac580e96f56777960e7/users/fb8e1abba297435d8a99fe8bf007597d/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.413602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1249a25-9730-4a49-98b7-7c496de2478e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54467f07001e4ac580e96f56777960e7, user_id=fb8e1abba297435d8a99fe8bf007597d, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.416567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1249a25-9730-4a49-98b7-7c496de2478e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.427426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1061/2094] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:11 2022] PUT /identity/v3/projects/54467f07001e4ac580e96f56777960e7/users/fb8e1abba297435d8a99fe8bf007597d/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:11.427963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.428858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.429135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.429465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.438378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.438883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.439232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.468286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] MFA Rules not processed for user `fb8e1abba297435d8a99fe8bf007597d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:11.469534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:11.490245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:11.508662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.510387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.522137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1034/2095] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:11.555077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1062/2096] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:11.561857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.570308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f1e0-2cc5-4db1-a903-3aeb22994ff7 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.632937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.633628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.633944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.634252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.679931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:11.703260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.704592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.714763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1035/2097] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:11.735145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:11.736703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:11.749115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef4307-43be-437a-a81c-8b138a6e2a08 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:11.753022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:11.753826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:11.754175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:11.754547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:11.812284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:11.813751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:11.823104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1063/2098] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:14.381934 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:14.392062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4bff-3397-4bfc-bd0b-9e8476a2a7e8 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:14.397311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:14.398171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:14.398633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:14.398907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:14.464910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:14.466507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:14.475774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1036/2099] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:15.141400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:15.150097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07101fc-5ff4-4df1-a363-86ed3645eab9 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:15.154471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94417634-88de-4e70-ac5e-9ef1559ac118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:15.155074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94417634-88de-4e70-ac5e-9ef1559ac118 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:15.155349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94417634-88de-4e70-ac5e-9ef1559ac118 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:15.155620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94417634-88de-4e70-ac5e-9ef1559ac118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:15.165205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94417634-88de-4e70-ac5e-9ef1559ac118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:15.166689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94417634-88de-4e70-ac5e-9ef1559ac118 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:15.177115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1064/2100] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:15.993112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:15.994650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:16.058890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04a827-1d62-4080-bf0b-749318c5b530 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:16.063340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:16.063950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:16.064224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:16.064490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] PATH_INFO: `/v3/users/cf690560b43c488ba7921945ce1faf41` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:16.065643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf690560b43c488ba7921945ce1faf41)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:16.067545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:16.111910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf690560b43c488ba7921945ce1faf41'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.118587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf690560b43c488ba7921945ce1faf41'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.125362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf690560b43c488ba7921945ce1faf41'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.125970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf690560b43c488ba7921945ce1faf41'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.126567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf690560b43c488ba7921945ce1faf41'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.134346 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:16.145583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1037/2101] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:15 2022] DELETE /identity/v3/users/cf690560b43c488ba7921945ce1faf41 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:16.149989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:16.150457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:16.150558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:16.170772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:16.171956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:16.229632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1065/2102] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:16.239244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:16.240620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:16.301661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6822ba-a9cb-4f1e-9ab8-09a6d70de18c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:16.305493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:16.306130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:16.306406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:16.306700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:16.316978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:16.318509 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:16.328714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1038/2103] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:16.619143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:16.620845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:16.634757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394a992-9624-4509-aad3-a2f0b9ac8d67 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:16.641611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:16.642355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:16.642709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:16.643077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] PATH_INFO: `/v3/projects/9532ceafd6ab4f418964f0599804beaa` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:16.644490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] RBAC: Authorizing `identity:delete_project(project_id=9532ceafd6ab4f418964f0599804beaa)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:16.646842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:16.704741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:16.712776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9532ceafd6ab4f418964f0599804beaa', 'request_id': 'req-1db11750-1681-433f-94c9-95179b8462de'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.718885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9532ceafd6ab4f418964f0599804beaa', 'request_id': 'req-1db11750-1681-433f-94c9-95179b8462de'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.726852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9532ceafd6ab4f418964f0599804beaa', 'request_id': 'req-1db11750-1681-433f-94c9-95179b8462de'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.738541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] The token cache is being invalidate because project 9532ceafd6ab4f418964f0599804beaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:16.739463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.740224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.741836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1066/2104] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:16 2022] DELETE /identity/v3/projects/9532ceafd6ab4f418964f0599804beaa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:16.745913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:16.756244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451498b-afbf-4ad9-8767-540115188090 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:16.811966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:16.812631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:16.812905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:16.813172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] PATH_INFO: `/v3/users/df3514ac980344e2b05978eaef1c2c60` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:16.814303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=df3514ac980344e2b05978eaef1c2c60)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:16.816242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:16.860812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df3514ac980344e2b05978eaef1c2c60'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.867690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df3514ac980344e2b05978eaef1c2c60'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.874682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df3514ac980344e2b05978eaef1c2c60'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.875289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df3514ac980344e2b05978eaef1c2c60'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.875927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df3514ac980344e2b05978eaef1c2c60'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:16.892384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1039/2105] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:16 2022] DELETE /identity/v3/users/df3514ac980344e2b05978eaef1c2c60 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:17.128269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.130132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:17.156835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db11750-1681-433f-94c9-95179b8462de admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.205728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.207126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:17.223128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:17.228822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.229954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.230388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.230790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] PATH_INFO: `/v3/users/34403fa224b04c13868d64b63231ad7c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.232067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34403fa224b04c13868d64b63231ad7c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.234070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.279937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31b2ce-a2d3-4fbd-bfeb-416db27995bf admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.285131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34403fa224b04c13868d64b63231ad7c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.285417 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.286111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.286396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.286675 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] PATH_INFO: `/v3/projects/71e3d271ed0a47b28c56d9db3660923e` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.287961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e3d271ed0a47b28c56d9db3660923e)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.290044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.293166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34403fa224b04c13868d64b63231ad7c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.293821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34403fa224b04c13868d64b63231ad7c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.294581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34403fa224b04c13868d64b63231ad7c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.302625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-049182a6-f815-40c1-ae8b-054ded8c8a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34403fa224b04c13868d64b63231ad7c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.321055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1067/2106] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:17 2022] DELETE /identity/v3/users/34403fa224b04c13868d64b63231ad7c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:17.325322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.326155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.326432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.339357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e3d271ed0a47b28c56d9db3660923e', 'request_id': 'req-224c90a4-e5c1-41c4-adf3-81c964af60e7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.344345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:17.345372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e3d271ed0a47b28c56d9db3660923e', 'request_id': 'req-224c90a4-e5c1-41c4-adf3-81c964af60e7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.345571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:17.354513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e3d271ed0a47b28c56d9db3660923e', 'request_id': 'req-224c90a4-e5c1-41c4-adf3-81c964af60e7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.365744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] The token cache is being invalidate because project 71e3d271ed0a47b28c56d9db3660923e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:17.366032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.366655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-224c90a4-e5c1-41c4-adf3-81c964af60e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.368365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1040/2107] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:17 2022] DELETE /identity/v3/projects/71e3d271ed0a47b28c56d9db3660923e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:17.374681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cfcbc-11dd-45d0-880b-e3a3c7b1a8db None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.375187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cfcbc-11dd-45d0-880b-e3a3c7b1a8db None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.375466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cfcbc-11dd-45d0-880b-e3a3c7b1a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.391607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:17.395178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4d4cfcbc-11dd-45d0-880b-e3a3c7b1a8db None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:17.396378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4d4cfcbc-11dd-45d0-880b-e3a3c7b1a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:17.412200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1068/2108] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:17.422253 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.423708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:17.465124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1041/2109] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:17.472841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1206f5-506b-4f41-bd3d-fc2d7d408f92 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.473125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1206f5-506b-4f41-bd3d-fc2d7d408f92 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.473432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1206f5-506b-4f41-bd3d-fc2d7d408f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.483913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff1e0-e318-4220-80e8-2bb3dcc212f7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.487687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.488426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.488698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.488967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.492151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-eb1206f5-506b-4f41-bd3d-fc2d7d408f92 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:17.493361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-eb1206f5-506b-4f41-bd3d-fc2d7d408f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:17.498828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.500274 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.501767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1042/2110] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:17.505906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1206f5-506b-4f41-bd3d-fc2d7d408f92 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.510093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1069/2111] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:17.518658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1206f5-506b-4f41-bd3d-fc2d7d408f92 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.523421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.524658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.524658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.524831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.525381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:17.525688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.527317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.539238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1043/2112] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:32:17.543350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.553286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfdaae-7fdd-457f-ac8c-a4bf19e05584 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.557970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.558568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.558847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.559120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.560250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.562276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.592545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1070/2113] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:17 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:17.596627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.607549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba9a88-5d32-44d4-9ba9-d713ad4bff00 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.649026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:17.723792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.724564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.724912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.725313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.726046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.730041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.758480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1044/2114] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:17 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:17.762666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.776564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb006b-2312-4b9e-a17e-b05f02ff6c53 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.785014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.786057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.786789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.787420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.789419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.791546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.806176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1071/2115] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:17.811721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.823685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.827372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34eb1-3d1f-42fd-9a27-07a5d41da3a1 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.833942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:17.852388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0248d75-5cbf-49bf-a57b-69522e046622 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.853429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.854320 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.854883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.855465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.856437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.858340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.858964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.859240 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.859510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] PATH_INFO: `/v3/projects/b4cd99269e0f494ea94269f9c7d53ac5` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.860657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.860970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4cd99269e0f494ea94269f9c7d53ac5)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.863279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.874352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1045/2116] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:17.879336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:17.896525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d0fe3-fe46-4765-8dcb-76a1b261a3ca admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:17.903435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d81d2-59c9-4428-9b2a-387422447b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:17.906575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d81d2-59c9-4428-9b2a-387422447b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.907080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d81d2-59c9-4428-9b2a-387422447b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.907477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d81d2-59c9-4428-9b2a-387422447b06 admin admin] PATH_INFO: `/v3/projects/704464b34c854520a2db9627c7ae480a/users/7635db9da4f5403cb12bd0040142638b/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.944537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d81d2-59c9-4428-9b2a-387422447b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704464b34c854520a2db9627c7ae480a, user_id=7635db9da4f5403cb12bd0040142638b, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:17.948152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d81d2-59c9-4428-9b2a-387422447b06 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:17.958924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4cd99269e0f494ea94269f9c7d53ac5', 'request_id': 'req-730e4112-6bce-4b01-8aee-f6fdf53f694c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.967351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4cd99269e0f494ea94269f9c7d53ac5', 'request_id': 'req-730e4112-6bce-4b01-8aee-f6fdf53f694c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.968794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1046/2117] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:17 2022] PUT /identity/v3/projects/704464b34c854520a2db9627c7ae480a/users/7635db9da4f5403cb12bd0040142638b/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:17.977570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4cd99269e0f494ea94269f9c7d53ac5', 'request_id': 'req-730e4112-6bce-4b01-8aee-f6fdf53f694c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.980832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:17.981178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:17.981570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:17.991813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] The token cache is being invalidate because project b4cd99269e0f494ea94269f9c7d53ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:17.992717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.993694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:17.996661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1072/2118] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:17 2022] DELETE /identity/v3/projects/b4cd99269e0f494ea94269f9c7d53ac5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:18.000836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.002590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:18.046932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] MFA Rules not processed for user `7635db9da4f5403cb12bd0040142638b`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:18.048183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:18.072483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:18.121378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e4112-6bce-4b01-8aee-f6fdf53f694c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.152159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1047/2119] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:18.158598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.159996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:18.221471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.222605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.222979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.223466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] PATH_INFO: `/v3/users/94dfb82a4e864d6f8d1526696f940f06` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.225214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94dfb82a4e864d6f8d1526696f940f06)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.227689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.266182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594453f-0688-4191-bf6f-5732ad65f244 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.272002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.272645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.272925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.273240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.274043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.276560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.294219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:18.297218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94dfb82a4e864d6f8d1526696f940f06'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.310361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94dfb82a4e864d6f8d1526696f940f06'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.311134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94dfb82a4e864d6f8d1526696f940f06'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.311803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94dfb82a4e864d6f8d1526696f940f06'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.314753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1048/2120] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:18 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:18.319459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.321840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94dfb82a4e864d6f8d1526696f940f06'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.323472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:18.361597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1073/2121] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:17 2022] DELETE /identity/v3/users/94dfb82a4e864d6f8d1526696f940f06 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:18.412303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d52c7d-0898-443f-8f72-a863f98b5da0 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.476891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.477775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.478098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.478474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.479465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.481560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.509946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1049/2122] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:18 2022] POST /identity/v3/users => generated 300 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:18.515218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.529399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376c39b-307b-4c6c-aa18-672e6aa0cfa4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.537576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.538203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.538629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.538763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.539423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.542870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.559023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1074/2123] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:18.560234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.574339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81327ff-4590-489d-9963-ba3506e3bc28 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.583955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.584739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.585087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.585422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] PATH_INFO: `/v3/projects/7a04c83fe0f2429c9aa7cf3e2bed36ec/users/e28fc8e8fce9461a852fa4625bbbeb94/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.609281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a04c83fe0f2429c9aa7cf3e2bed36ec, user_id=e28fc8e8fce9461a852fa4625bbbeb94, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.613937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.635206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1050/2124] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:18 2022] PUT /identity/v3/projects/7a04c83fe0f2429c9aa7cf3e2bed36ec/users/e28fc8e8fce9461a852fa4625bbbeb94/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:18.640563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.655142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedb180-4a0a-4c1c-a819-8380772ae4f9 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.674724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.675519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:18.753119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:18.761545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.762456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.763251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.763883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.764831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.766542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.777365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1075/2125] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:18.782183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.785693 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a47e52-8786-45d6-bae3-3b8845f01b57 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.792213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.792365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10940adc-1c2c-41eb-bb8a-19423e794eec admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:18.792884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.793173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.794064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] PATH_INFO: `/v3/projects/3510c833e83649228a5df27d65e7228c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.795770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3510c833e83649228a5df27d65e7228c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.797740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.797912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db66362-25ff-4f46-a471-40027070cb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:18.798526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66362-25ff-4f46-a471-40027070cb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.798935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66362-25ff-4f46-a471-40027070cb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.799217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66362-25ff-4f46-a471-40027070cb0e admin admin] PATH_INFO: `/v3/projects/7a04c83fe0f2429c9aa7cf3e2bed36ec/users/e28fc8e8fce9461a852fa4625bbbeb94/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.801204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db66362-25ff-4f46-a471-40027070cb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a04c83fe0f2429c9aa7cf3e2bed36ec, user_id=e28fc8e8fce9461a852fa4625bbbeb94, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:18.803912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db66362-25ff-4f46-a471-40027070cb0e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:18.807876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:18.834890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1076/2126] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:18 2022] PUT /identity/v3/projects/7a04c83fe0f2429c9aa7cf3e2bed36ec/users/e28fc8e8fce9461a852fa4625bbbeb94/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:18.837330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.837818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.838937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.864226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3510c833e83649228a5df27d65e7228c', 'request_id': 'req-495fb55f-dd46-408a-b868-0ba001b2b562'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.869254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] MFA Rules not processed for user `e28fc8e8fce9461a852fa4625bbbeb94`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:18.870808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:18.871009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3510c833e83649228a5df27d65e7228c', 'request_id': 'req-495fb55f-dd46-408a-b868-0ba001b2b562'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.881203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3510c833e83649228a5df27d65e7228c', 'request_id': 'req-495fb55f-dd46-408a-b868-0ba001b2b562'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.896994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] The token cache is being invalidate because project 3510c833e83649228a5df27d65e7228c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:18.897315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.898408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-495fb55f-dd46-408a-b868-0ba001b2b562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:18.900051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1051/2127] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:18 2022] DELETE /identity/v3/projects/3510c833e83649228a5df27d65e7228c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:18.906224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6faa3-1c91-49af-b470-ae3ec26d37f7 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:18.906534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6faa3-1c91-49af-b470-ae3ec26d37f7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:18.906834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6faa3-1c91-49af-b470-ae3ec26d37f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:18.929962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-cdc6faa3-1c91-49af-b470-ae3ec26d37f7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:18.931631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cdc6faa3-1c91-49af-b470-ae3ec26d37f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:18.965462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1077/2128] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:18.978560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:18.985219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:19.009606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1052/2129] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:19.019834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe43ae1-679f-4d40-ae1e-37152ef99044 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.020132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe43ae1-679f-4d40-ae1e-37152ef99044 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.020393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe43ae1-679f-4d40-ae1e-37152ef99044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.050059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ebe43ae1-679f-4d40-ae1e-37152ef99044 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:19.052603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ebe43ae1-679f-4d40-ae1e-37152ef99044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:19.054210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5688081-e3aa-48db-a1c1-c507b3926924 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.057874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.058460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.058746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.059019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.063720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1053/2130] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:19.068342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe43ae1-679f-4d40-ae1e-37152ef99044 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.073406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.074869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.084416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe43ae1-679f-4d40-ae1e-37152ef99044 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.086441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1078/2131] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:19.092422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.093832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.098906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.099546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.100547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:19.101111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.103312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.131159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1054/2132] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:19.132729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.146216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624a114-e100-4403-a19d-d005173619ee service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.153069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.154383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.154737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.155105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.155830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.157910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.210156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1079/2133] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:19 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:19.214215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.237437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbacab-198d-4f85-8175-c4ef2d6be835 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.330515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.331473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.331552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.331990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.333051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.335624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.342661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.345704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:19.369647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da14a69-aa15-49e4-9026-341ccb144ca5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.375518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1055/2134] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:19 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:19.376422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.378016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.378617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.378891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.379173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] PATH_INFO: `/v3/users/61038258e95144a8bc1a424192c3dbc1` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.382193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=61038258e95144a8bc1a424192c3dbc1)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.385692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.389789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf4b35-a2a0-45c6-b1fd-2335c3137667 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.395632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.396249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.396526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.396796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.398000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.401708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.403186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:19.423290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1056/2135] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:19.425460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.441949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1e2a4-2654-42b1-8d2b-0a89f75eceb1 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.457424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.458246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.458880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.459222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.460219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.462874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.484379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:19.504063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1057/2136] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:19.506014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61038258e95144a8bc1a424192c3dbc1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:19.508016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.521233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad69d2e-fe74-4158-b694-a370b4567573 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.524313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61038258e95144a8bc1a424192c3dbc1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:19.525435 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61038258e95144a8bc1a424192c3dbc1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:19.527973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8c869-3af6-4794-a513-baa6eb9fdcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.528413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61038258e95144a8bc1a424192c3dbc1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:19.530038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8c869-3af6-4794-a513-baa6eb9fdcc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.530405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8c869-3af6-4794-a513-baa6eb9fdcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.530730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8c869-3af6-4794-a513-baa6eb9fdcc8 admin admin] PATH_INFO: `/v3/projects/6e2a8e16ccc4426dbda0865ef0208bda/users/2e63e408731348858017070ddc558fd9/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.543244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-d9827fe2-91f8-4e5e-9fae-a26c36455b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61038258e95144a8bc1a424192c3dbc1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:19.553616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8c869-3af6-4794-a513-baa6eb9fdcc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2a8e16ccc4426dbda0865ef0208bda, user_id=2e63e408731348858017070ddc558fd9, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.559429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8c869-3af6-4794-a513-baa6eb9fdcc8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.582019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1080/2137] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:19 2022] DELETE /identity/v3/users/61038258e95144a8bc1a424192c3dbc1 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:19.583593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.583891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.584174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.608228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:19.609450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:19.610672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1058/2138] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:19 2022] PUT /identity/v3/projects/6e2a8e16ccc4426dbda0865ef0208bda/users/2e63e408731348858017070ddc558fd9/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:19.626090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.626562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.627005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.702744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] MFA Rules not processed for user `2e63e408731348858017070ddc558fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:19.704239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:19.772020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1081/2139] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:19.779714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.782825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:19.848504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1059/2140] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:19.867727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:19.870275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:19.907221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ca5d3-dba2-4d7d-ad8e-02db73f848b8 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.910976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.911589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.911864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.912274 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.923189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.924619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.935807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1082/2141] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:19.936605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f749463-476b-4d7b-8563-4c2b4887f30d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:19.941031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9be8c8c5f7f6417e9ab19ef5bce0f060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:19.941740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:19.942015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:19.942283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:19.953560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:19.955090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:19.965036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1060/2142] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:20.082828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:20.084458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:20.105326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f30cea-4894-4b6a-aec7-8063e3011908 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:20.113587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:20.114533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:20.115119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:20.115514 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] PATH_INFO: `/v3/users/7635db9da4f5403cb12bd0040142638b` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:20.132566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=7635db9da4f5403cb12bd0040142638b)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:20.135278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:20.250779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7635db9da4f5403cb12bd0040142638b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.261314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7635db9da4f5403cb12bd0040142638b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.262126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7635db9da4f5403cb12bd0040142638b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.263441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7635db9da4f5403cb12bd0040142638b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.265057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:20.279998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-7bc499ab-1a28-41c4-a9f9-92700945a4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7635db9da4f5403cb12bd0040142638b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.296638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:20.298457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:20.313522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1083/2143] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:20 2022] DELETE /identity/v3/users/7635db9da4f5403cb12bd0040142638b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:20.317335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:20.317684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:20.318467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:20.341230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3225c87-6f99-4387-8f64-16e85b45aac7 service glance] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:20.341747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:20.343075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:20.406140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:20.418176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:20.418809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:20.419111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:20.419381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] PATH_INFO: `/v3/projects/197f506933244ff1a5d47d8e84255d0e` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:20.420501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197f506933244ff1a5d47d8e84255d0e)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:20.422737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:20.423363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1084/2144] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:20.433190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:20.435750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:20.482012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197f506933244ff1a5d47d8e84255d0e', 'request_id': 'req-0b1cfeec-7f31-4dda-8816-f837eaa63e49'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.488781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197f506933244ff1a5d47d8e84255d0e', 'request_id': 'req-0b1cfeec-7f31-4dda-8816-f837eaa63e49'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.499238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197f506933244ff1a5d47d8e84255d0e', 'request_id': 'req-0b1cfeec-7f31-4dda-8816-f837eaa63e49'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.511979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] The token cache is being invalidate because project 197f506933244ff1a5d47d8e84255d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:20.512273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.513745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.515260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1061/2145] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:20 2022] DELETE /identity/v3/projects/197f506933244ff1a5d47d8e84255d0e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:20.518859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:20.519475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dbe49-9c89-48e6-b80f-ec30e3d83790 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:20.520395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:20.598084 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:20.599248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:20.599741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:20.600815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:20.603273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:20.636449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cfeec-7f31-4dda-8816-f837eaa63e49 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:20.645519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:20.646497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:20.647219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:20.647552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] PATH_INFO: `/v3/users/7bf3b6da1fa8416e9ff66490983808bd` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:20.648737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf3b6da1fa8416e9ff66490983808bd)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:20.651558 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:20.675899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:20.702166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:20.703757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:20.712175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf3b6da1fa8416e9ff66490983808bd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.714201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1085/2146] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:20.726883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf3b6da1fa8416e9ff66490983808bd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.737569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf3b6da1fa8416e9ff66490983808bd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.738271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf3b6da1fa8416e9ff66490983808bd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.738968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf3b6da1fa8416e9ff66490983808bd'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:20.758306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1062/2147] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:20 2022] DELETE /identity/v3/users/7bf3b6da1fa8416e9ff66490983808bd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:21.147667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:21.149218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:21.160278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:21.161843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:21.163275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459990-1ede-490b-8a58-c5d2e583de0e service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:21.168630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:21.240935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:21.241714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:21.246786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8f63d-c19f-49ad-a1b7-9989eef79f30 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:21.250618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:21.251079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] PATH_INFO: `/v3/projects/704464b34c854520a2db9627c7ae480a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:21.252404 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] RBAC: Authorizing `identity:delete_project(project_id=704464b34c854520a2db9627c7ae480a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:21.254408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:21.254617 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:21.255375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:21.255764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:21.256265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] PATH_INFO: `/v3/projects/597c02e5db224da1b11e09136b11ea2b` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:21.257672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=597c02e5db224da1b11e09136b11ea2b)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:21.259847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:21.320275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704464b34c854520a2db9627c7ae480a', 'request_id': 'req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.327164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704464b34c854520a2db9627c7ae480a', 'request_id': 'req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.330545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '597c02e5db224da1b11e09136b11ea2b', 'request_id': 'req-5d280b1a-df5f-47b8-af45-bbd7010398f8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.337838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704464b34c854520a2db9627c7ae480a', 'request_id': 'req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.338041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '597c02e5db224da1b11e09136b11ea2b', 'request_id': 'req-5d280b1a-df5f-47b8-af45-bbd7010398f8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.345219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '597c02e5db224da1b11e09136b11ea2b', 'request_id': 'req-5d280b1a-df5f-47b8-af45-bbd7010398f8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.351256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] The token cache is being invalidate because project 704464b34c854520a2db9627c7ae480a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:21.351712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.352413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.354339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1086/2148] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:21 2022] DELETE /identity/v3/projects/704464b34c854520a2db9627c7ae480a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:21.358189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:21.358441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] The token cache is being invalidate because project 597c02e5db224da1b11e09136b11ea2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:21.358732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.359374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.359480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:21.360865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1063/2149] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:21 2022] DELETE /identity/v3/projects/597c02e5db224da1b11e09136b11ea2b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:21.364430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:21.365754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:21.429416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142c1a4-d0af-4f7c-8965-a2784fe1c11a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:21.434557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:21.435500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d280b1a-df5f-47b8-af45-bbd7010398f8 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:21.435936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:21.436312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:21.436657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] PATH_INFO: `/v3/users/e28fc8e8fce9461a852fa4625bbbeb94` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:21.438670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e28fc8e8fce9461a852fa4625bbbeb94)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:21.440877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:21.451391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:21.452614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:21.453082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:21.453550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] PATH_INFO: `/v3/users/fb8e1abba297435d8a99fe8bf007597d` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:21.455721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8e1abba297435d8a99fe8bf007597d)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:21.458296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:21.473772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:21.509040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:21.522233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8e1abba297435d8a99fe8bf007597d'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.525184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e28fc8e8fce9461a852fa4625bbbeb94'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.532890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8e1abba297435d8a99fe8bf007597d'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.538871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e28fc8e8fce9461a852fa4625bbbeb94'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.539697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e28fc8e8fce9461a852fa4625bbbeb94'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.540445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e28fc8e8fce9461a852fa4625bbbeb94'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.542003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8e1abba297435d8a99fe8bf007597d'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.542741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8e1abba297435d8a99fe8bf007597d'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.543503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8e1abba297435d8a99fe8bf007597d'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.550235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e28fc8e8fce9461a852fa4625bbbeb94'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:21.569623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1064/2150] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:21 2022] DELETE /identity/v3/users/fb8e1abba297435d8a99fe8bf007597d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:21.572749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1087/2151] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:21 2022] DELETE /identity/v3/users/e28fc8e8fce9461a852fa4625bbbeb94 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:21.966811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:21.968276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:21.973546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:21.977433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.101809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9abe0a-3244-4139-972f-a2be4a9a4ad5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.104498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ebffa-58da-41e5-837b-43d0a175342e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.109412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.110108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.110333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.110605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] PATH_INFO: `/v3/projects/54467f07001e4ac580e96f56777960e7` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.111744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54467f07001e4ac580e96f56777960e7)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.113724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.118177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.119114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.119485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.119840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] PATH_INFO: `/v3/projects/7a04c83fe0f2429c9aa7cf3e2bed36ec` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.121493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a04c83fe0f2429c9aa7cf3e2bed36ec)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.124040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.171770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54467f07001e4ac580e96f56777960e7', 'request_id': 'req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.180487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54467f07001e4ac580e96f56777960e7', 'request_id': 'req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.190170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54467f07001e4ac580e96f56777960e7', 'request_id': 'req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.196086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a04c83fe0f2429c9aa7cf3e2bed36ec', 'request_id': 'req-e49dc3df-3dfe-4b6e-953e-b940239cd097'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.201924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a04c83fe0f2429c9aa7cf3e2bed36ec', 'request_id': 'req-e49dc3df-3dfe-4b6e-953e-b940239cd097'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.203962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] The token cache is being invalidate because project 54467f07001e4ac580e96f56777960e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:22.204434 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.205311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-b7c463fa-caa8-437f-bca0-2a08c66ac0a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.206925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1088/2152] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:21 2022] DELETE /identity/v3/projects/54467f07001e4ac580e96f56777960e7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:22.210726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a04c83fe0f2429c9aa7cf3e2bed36ec', 'request_id': 'req-e49dc3df-3dfe-4b6e-953e-b940239cd097'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.212178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77851eb7-d49f-4f2a-9d0c-c7f6536b73a5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.212457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77851eb7-d49f-4f2a-9d0c-c7f6536b73a5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.212728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77851eb7-d49f-4f2a-9d0c-c7f6536b73a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.231220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] The token cache is being invalidate because project 7a04c83fe0f2429c9aa7cf3e2bed36ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:22.232278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.232976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e49dc3df-3dfe-4b6e-953e-b940239cd097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:22.234750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1065/2153] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:21 2022] DELETE /identity/v3/projects/7a04c83fe0f2429c9aa7cf3e2bed36ec => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:22.234881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-77851eb7-d49f-4f2a-9d0c-c7f6536b73a5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:22.235863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-77851eb7-d49f-4f2a-9d0c-c7f6536b73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.239064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d892109-aaa1-4cf2-a6e5-7fdd58fe1e78 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.239355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d892109-aaa1-4cf2-a6e5-7fdd58fe1e78 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.239636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d892109-aaa1-4cf2-a6e5-7fdd58fe1e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.248558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:22.259949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1d892109-aaa1-4cf2-a6e5-7fdd58fe1e78 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:22.261144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1d892109-aaa1-4cf2-a6e5-7fdd58fe1e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.301080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:22.301296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1089/2154] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:22.309410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013e4bc-07d9-47e8-b964-54e9326a6c6a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.310336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013e4bc-07d9-47e8-b964-54e9326a6c6a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.310646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013e4bc-07d9-47e8-b964-54e9326a6c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.324812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1066/2155] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:22.330835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a013e4bc-07d9-47e8-b964-54e9326a6c6a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:22.332017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a013e4bc-07d9-47e8-b964-54e9326a6c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.334930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1112-f515-4db0-8e22-643958c67fc7 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.335387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1112-f515-4db0-8e22-643958c67fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.337262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1112-f515-4db0-8e22-643958c67fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.341311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1090/2156] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:22.344622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013e4bc-07d9-47e8-b964-54e9326a6c6a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.355571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013e4bc-07d9-47e8-b964-54e9326a6c6a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.361780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.362634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.363092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-7c5e1112-f515-4db0-8e22-643958c67fc7 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:22.363600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.364055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.364306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7c5e1112-f515-4db0-8e22-643958c67fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.365077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:22.365660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.367619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.377125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1067/2157] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:22.378431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e1112-f515-4db0-8e22-643958c67fc7 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.380713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1091/2158] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:22.385424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.387771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e1112-f515-4db0-8e22-643958c67fc7 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.392765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.393418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.394192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.394192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.394613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:22.394918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.396536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.397945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d6682-336a-44c3-b28b-7c199819ce5f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.403001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.403720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.404106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.404379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.405224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.407078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.409830 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1068/2159] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:22.414239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.424605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30e067-de44-48f3-b186-9b236084441b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.430479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.431066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.431353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.431628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.432317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.434285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.435543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1092/2160] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:22.439653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.463478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1069/2161] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:22.467666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.507477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a41ead-49ed-430d-befd-f931dfe67d10 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.536785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6d042-1a4a-4a3b-a4ce-0f0d2a9c42c0 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.542235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.542834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.543110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.543380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.544129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.547764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.566423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.567458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.568027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.568490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.569343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.571538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.577786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1070/2162] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/users => generated 281 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:22.581927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.593149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c951-9577-4fd4-aa3f-bfd01aedf244 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.596738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1093/2163] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/users => generated 283 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:22.600840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.604005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.604612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.604895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.605171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.605849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.607483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.610222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1225e-cf48-4468-843d-2dcbaee6a47a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.614739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.615323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.615599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.615869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.616497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.620821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1071/2164] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:22.620968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.623659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.631292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1094/2165] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:22.632399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.635131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3957752-0b97-459e-9dd7-5dba5ecaf594 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.638071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:22.643858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.644501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.644774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.645041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.645712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.646517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f607331-b68a-47d0-87ea-c4f92f5d5e92 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.647329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.651417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.652012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.652290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.652559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.653196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.654836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.658441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1072/2166] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:22.662407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.665614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1095/2167] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:22.669602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.671893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee7e82-8436-4df4-a40d-8ceff83ed630 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.676376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874618f-ebf0-4b7e-bcfe-6900db054693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.677001 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874618f-ebf0-4b7e-bcfe-6900db054693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.677288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874618f-ebf0-4b7e-bcfe-6900db054693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.677577 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874618f-ebf0-4b7e-bcfe-6900db054693 admin admin] PATH_INFO: `/v3/projects/0975263f54a54516bdcef5bffda6f6e8/users/da5030753dbc4dc2a0bcc118e31fe610/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.678912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56235853-37c5-4e7b-a91d-5f7cf2f0ec75 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.685301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bfd33-7a96-4974-b40f-c8363414da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.686400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bfd33-7a96-4974-b40f-c8363414da76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.689925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bfd33-7a96-4974-b40f-c8363414da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.690292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bfd33-7a96-4974-b40f-c8363414da76 admin admin] PATH_INFO: `/v3/projects/4cae2733cd51406b9e572cc98b091eac/users/52e0a145380148a1bbeb3562991843e8/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.694973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874618f-ebf0-4b7e-bcfe-6900db054693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0975263f54a54516bdcef5bffda6f6e8, user_id=da5030753dbc4dc2a0bcc118e31fe610, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.697370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874618f-ebf0-4b7e-bcfe-6900db054693 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.710761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bfd33-7a96-4974-b40f-c8363414da76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cae2733cd51406b9e572cc98b091eac, user_id=52e0a145380148a1bbeb3562991843e8, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.713129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1073/2168] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:22 2022] PUT /identity/v3/projects/0975263f54a54516bdcef5bffda6f6e8/users/da5030753dbc4dc2a0bcc118e31fe610/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:22.713581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bfd33-7a96-4974-b40f-c8363414da76 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.724203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.724506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.724773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.728741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1096/2169] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:22 2022] PUT /identity/v3/projects/4cae2733cd51406b9e572cc98b091eac/users/52e0a145380148a1bbeb3562991843e8/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:22.740365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.740653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.740930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.758285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] MFA Rules not processed for user `da5030753dbc4dc2a0bcc118e31fe610`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:22.759645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.770410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] MFA Rules not processed for user `52e0a145380148a1bbeb3562991843e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:22.771572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.798983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:22.834818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1074/2170] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:22.843603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1097/2171] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:22.854867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.856276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.869405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:22.870831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:22.914250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6db85-659c-4eb4-9351-dcddecd9d635 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.917872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.918472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.918748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.919578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.927657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.929095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.929449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f014a37-75fe-4b00-9bde-f6b406db5e80 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:22.934028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc145e79-9ebd-43fc-a4cc-e558238de665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:22.934647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc145e79-9ebd-43fc-a4cc-e558238de665 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:22.935567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc145e79-9ebd-43fc-a4cc-e558238de665 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:22.935717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc145e79-9ebd-43fc-a4cc-e558238de665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:22.939436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1075/2172] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:22.944881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc145e79-9ebd-43fc-a4cc-e558238de665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:22.946236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc145e79-9ebd-43fc-a4cc-e558238de665 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:22.964145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1098/2173] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:22.979815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:23.050527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea8b34-d921-4a44-9240-f0499b0b0460 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:23.055408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:23.056043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:23.056326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:23.056602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] PATH_INFO: `/v3/users/da5030753dbc4dc2a0bcc118e31fe610` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:23.057755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5030753dbc4dc2a0bcc118e31fe610)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:23.060322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:23.113732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5030753dbc4dc2a0bcc118e31fe610'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.120654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5030753dbc4dc2a0bcc118e31fe610'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.128949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5030753dbc4dc2a0bcc118e31fe610'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.129606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5030753dbc4dc2a0bcc118e31fe610'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.130225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5030753dbc4dc2a0bcc118e31fe610'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.141999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:23.150375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1076/2174] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:22 2022] DELETE /identity/v3/users/da5030753dbc4dc2a0bcc118e31fe610 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:23.154990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:23.155283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:23.155561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:23.176269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:23.177620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:23.259283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:23.279823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1099/2175] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:23.286579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:23.288227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:23.363520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd3e5-a59e-4bc1-a73e-aebcacd4a6f1 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:23.367246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90950584-2a38-4e4c-9390-8a91a32400e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:23.367856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950584-2a38-4e4c-9390-8a91a32400e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:23.368139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950584-2a38-4e4c-9390-8a91a32400e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:23.368418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950584-2a38-4e4c-9390-8a91a32400e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:23.378073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90950584-2a38-4e4c-9390-8a91a32400e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:23.379572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90950584-2a38-4e4c-9390-8a91a32400e0 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:23.390814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1077/2176] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:23.899478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:23.901053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:23.913821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11b648-88ef-4d59-8b69-16dae6158510 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:23.918782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:23.920104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:23.920345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:23.920612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] PATH_INFO: `/v3/projects/0975263f54a54516bdcef5bffda6f6e8` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:23.921799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0975263f54a54516bdcef5bffda6f6e8)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:23.923715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:23.973359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0975263f54a54516bdcef5bffda6f6e8', 'request_id': 'req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.978975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0975263f54a54516bdcef5bffda6f6e8', 'request_id': 'req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.986768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0975263f54a54516bdcef5bffda6f6e8', 'request_id': 'req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.998885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] The token cache is being invalidate because project 0975263f54a54516bdcef5bffda6f6e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:23.999192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:23.999895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e79fdd86-cf8d-4085-8a3d-da7ddb736b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:24.008935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1100/2177] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:23 2022] DELETE /identity/v3/projects/0975263f54a54516bdcef5bffda6f6e8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:24.009578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeaf73-a9c4-4d56-afdf-9db2a7224a11 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.009578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeaf73-a9c4-4d56-afdf-9db2a7224a11 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.009578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeaf73-a9c4-4d56-afdf-9db2a7224a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.031667 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5cfeaf73-a9c4-4d56-afdf-9db2a7224a11 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:24.032911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5cfeaf73-a9c4-4d56-afdf-9db2a7224a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:24.097562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1078/2178] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:32:24.106401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7ac56-1001-49a8-9b89-4017d86ba1eb None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.107272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7ac56-1001-49a8-9b89-4017d86ba1eb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.107551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7ac56-1001-49a8-9b89-4017d86ba1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.127304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ddc7ac56-1001-49a8-9b89-4017d86ba1eb None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:24.128649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ddc7ac56-1001-49a8-9b89-4017d86ba1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:24.137044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1101/2179] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:24.141396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeaf73-a9c4-4d56-afdf-9db2a7224a11 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:24.151013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeaf73-a9c4-4d56-afdf-9db2a7224a11 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:24.156047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:24.156654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.156937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.157211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.157892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:24.158200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:24.159828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:24.175791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1079/2180] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:24.177255 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7ac56-1001-49a8-9b89-4017d86ba1eb None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:24.188238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7ac56-1001-49a8-9b89-4017d86ba1eb None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:24.193823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:24.194461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.194743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.195009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.195645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:24.197704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:24.229787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1102/2181] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:24 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:24.234629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:24.244647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a914eb2-e0f0-4d5d-8c86-e6a505673d57 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:24.304503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:24.305113 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.305554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.305829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.306464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:24.308567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:24.331364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1080/2182] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:24 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:32:24.335723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:24.348992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6226370-23fd-44e2-9b7e-179923fe0138 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:24.355984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:24.357608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.357608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.357898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.358549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:24.360149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:24.373447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1103/2183] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:24.376484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:24.388106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701408-9b44-4dfa-9e1e-ce7b35a5c426 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:24.395612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d2938-245c-4102-a327-a7379c613a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:24.396345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d2938-245c-4102-a327-a7379c613a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.396696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d2938-245c-4102-a327-a7379c613a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.397041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d2938-245c-4102-a327-a7379c613a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.397855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d2938-245c-4102-a327-a7379c613a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:24.403142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d2938-245c-4102-a327-a7379c613a66 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:24.422560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1081/2184] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:32:24.425678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:24.455675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac4c8a-5b77-4ead-8c2d-b22510e8181b admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:24.462615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:24.476701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.476701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.476701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] PATH_INFO: `/v3/projects/4d5dd788aafe4cd1841c71885bf824eb/users/bc6edd11a4164dfd96eb6d0fe9d44bcb/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.487379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5dd788aafe4cd1841c71885bf824eb, user_id=bc6edd11a4164dfd96eb6d0fe9d44bcb, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:24.490870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:24.516559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1104/2185] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:24 2022] PUT /identity/v3/projects/4d5dd788aafe4cd1841c71885bf824eb/users/bc6edd11a4164dfd96eb6d0fe9d44bcb/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:32:24.527034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.527355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.527679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.540055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:24.567729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] MFA Rules not processed for user `bc6edd11a4164dfd96eb6d0fe9d44bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:24.569040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:24.674572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1082/2186] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:24.685363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:24.687307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:24.793727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd4737-cc85-4385-9258-21a6a78a8264 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:24.798717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:24.799767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:24.800180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:24.800387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:24.815182 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:24.816685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:24.864889 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1105/2187] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:27.404466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:27.405978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:27.478576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487597a-0687-4328-9cb1-fc120fc822d2 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:27.483710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:27.484532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:27.484988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:27.485438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] PATH_INFO: `/v3/users/94473332fd86422a9f1a1eed466f5a5a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:27.486606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=94473332fd86422a9f1a1eed466f5a5a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:27.488633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:27.539220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94473332fd86422a9f1a1eed466f5a5a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.547008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94473332fd86422a9f1a1eed466f5a5a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.555612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94473332fd86422a9f1a1eed466f5a5a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.556238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94473332fd86422a9f1a1eed466f5a5a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.556833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6dbf8d58-5f18-4f2e-91e9-2b974ec60d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94473332fd86422a9f1a1eed466f5a5a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.574718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1083/2188] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:27 2022] DELETE /identity/v3/users/94473332fd86422a9f1a1eed466f5a5a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:27.750716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:27.761742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de5a2-8827-47a0-8968-74603af6d0c5 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:27.781023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:27.820165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:27.820736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:27.821027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:27.821312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] PATH_INFO: `/v3/projects/28eda660c33645f8a4428987084ab934` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:27.822436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28eda660c33645f8a4428987084ab934)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:27.824354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:27.869021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28eda660c33645f8a4428987084ab934', 'request_id': 'req-131717f5-dceb-4a83-9ac3-1f95fd787632'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.875087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28eda660c33645f8a4428987084ab934', 'request_id': 'req-131717f5-dceb-4a83-9ac3-1f95fd787632'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.882171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28eda660c33645f8a4428987084ab934', 'request_id': 'req-131717f5-dceb-4a83-9ac3-1f95fd787632'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.893531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] The token cache is being invalidate because project 28eda660c33645f8a4428987084ab934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:27.893818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.894436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-131717f5-dceb-4a83-9ac3-1f95fd787632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:27.895929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1106/2189] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:27 2022] DELETE /identity/v3/projects/28eda660c33645f8a4428987084ab934 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:27.900848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4a49a-8938-4f17-9a72-ffd6711178dc None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:27.901134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4a49a-8938-4f17-9a72-ffd6711178dc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:27.901445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4a49a-8938-4f17-9a72-ffd6711178dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:27.920653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f0b4a49a-8938-4f17-9a72-ffd6711178dc None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:27.921890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f0b4a49a-8938-4f17-9a72-ffd6711178dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:27.930163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:27.983950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1084/2190] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:27.991758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679ecc5-ee3b-44b0-8459-30f2bb198be3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:27.992043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679ecc5-ee3b-44b0-8459-30f2bb198be3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:27.992317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679ecc5-ee3b-44b0-8459-30f2bb198be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.011992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-1679ecc5-ee3b-44b0-8459-30f2bb198be3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:28.013856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1679ecc5-ee3b-44b0-8459-30f2bb198be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:28.021611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1107/2191] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:28.025701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4a49a-8938-4f17-9a72-ffd6711178dc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:28.035936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4a49a-8938-4f17-9a72-ffd6711178dc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:28.040526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:28.041120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.041435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.041709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.042364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:32:28.042663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:28.044844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:28.056557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1085/2192] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:32:28.060973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679ecc5-ee3b-44b0-8459-30f2bb198be3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:28.070071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679ecc5-ee3b-44b0-8459-30f2bb198be3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:28.074747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:28.075286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.075563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.075834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.076470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:28.078409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:28.109929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1108/2193] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:32:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:28.111905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:28.121815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80a231-eb68-4f26-8759-705f59fd07a6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:28.176765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:28.177388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.177673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.177948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.178570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:28.180501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:28.202161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1086/2194] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:32:28 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:32:28.206318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:28.217163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e18d1-dfe2-488e-8eb1-94ce8b6242f4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:28.221871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:28.222739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.223018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.223294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.223936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:28.225561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:28.237395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1109/2195] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:28.239774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:28.249405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600779b-c964-4090-a847-2c7ae0bde2b9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:28.254216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf78285-969b-4de3-b90e-d3d7fa5d03a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:28.254857 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78285-969b-4de3-b90e-d3d7fa5d03a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.255133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78285-969b-4de3-b90e-d3d7fa5d03a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.255409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78285-969b-4de3-b90e-d3d7fa5d03a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.256050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf78285-969b-4de3-b90e-d3d7fa5d03a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:28.257763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf78285-969b-4de3-b90e-d3d7fa5d03a9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:28.269160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1087/2196] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:32:28.273482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:28.284191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebff81-1029-4efd-a8b6-0b6d801b3441 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:28.289220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:28.289985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.290333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.290677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] PATH_INFO: `/v3/projects/82c24c64f4c34921837eceb32e24a287/users/e6842565de354b0c85b04940d57c7432/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.308547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c24c64f4c34921837eceb32e24a287, user_id=e6842565de354b0c85b04940d57c7432, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:28.311610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:28.324683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1110/2197] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:32:28 2022] PUT /identity/v3/projects/82c24c64f4c34921837eceb32e24a287/users/e6842565de354b0c85b04940d57c7432/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:28.330363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae365-e7e2-43db-bb56-cfa74790cc02 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.330651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae365-e7e2-43db-bb56-cfa74790cc02 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.330926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae365-e7e2-43db-bb56-cfa74790cc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.350275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-873ae365-e7e2-43db-bb56-cfa74790cc02 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:28.351491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-873ae365-e7e2-43db-bb56-cfa74790cc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:28.410660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1088/2198] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:28.420432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:28.422292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:28.475382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:28.479340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661f811-afd4-4efd-913d-e095a45127aa admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:28.484063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:28.485045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:28.485673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:28.485935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:28.495392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:28.496818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:28.506258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1111/2199] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:32:32.865515 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a180f8-7d8e-4e92-8c43-b9a914f0cd5e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:32.866028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a180f8-7d8e-4e92-8c43-b9a914f0cd5e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:32.866148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a180f8-7d8e-4e92-8c43-b9a914f0cd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:32.894565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-06a180f8-7d8e-4e92-8c43-b9a914f0cd5e None None] MFA Rules not processed for user `e6842565de354b0c85b04940d57c7432`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:32.895690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-06a180f8-7d8e-4e92-8c43-b9a914f0cd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:32.904453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:32.977976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1089/2200] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:32.993315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:32.995398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:33.082269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e46957-935b-49ab-a7dd-fa25ee12307c service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:33.088023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:33.088653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:33.088932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:33.089207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:33.099902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:33.101855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:33.124487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1112/2201] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:34.489422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac218-1da2-4ca1-b41e-c77b376e508e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:34.489853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac218-1da2-4ca1-b41e-c77b376e508e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:34.490039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac218-1da2-4ca1-b41e-c77b376e508e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:34.510382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-194ac218-1da2-4ca1-b41e-c77b376e508e None None] MFA Rules not processed for user `e6842565de354b0c85b04940d57c7432`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:34.512253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-194ac218-1da2-4ca1-b41e-c77b376e508e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:34.519459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1090/2202] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:43.107311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=38 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:44.505330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=26 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:46.714180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:46.715580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:46.729634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d683-02db-4c84-a9e1-0beb2bfa44e9 service nova] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:46.734574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:46.735202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:46.735475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:46.736057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] PATH_INFO: `/v3/users/bc6edd11a4164dfd96eb6d0fe9d44bcb` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:46.737186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc6edd11a4164dfd96eb6d0fe9d44bcb)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:46.739145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:46.787385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc6edd11a4164dfd96eb6d0fe9d44bcb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:46.794842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc6edd11a4164dfd96eb6d0fe9d44bcb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:46.795674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc6edd11a4164dfd96eb6d0fe9d44bcb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:46.796286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc6edd11a4164dfd96eb6d0fe9d44bcb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:46.803392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc6edd11a4164dfd96eb6d0fe9d44bcb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:46.820994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1113/2203] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:32:46 2022] DELETE /identity/v3/users/bc6edd11a4164dfd96eb6d0fe9d44bcb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:46.825969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:46.826477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:46.826566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:46.847390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:32:46.848577 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:46.907807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1091/2204] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:32:46.918043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:46.919451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:46.977584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e11a5-e88b-4899-88fe-59f7565ca7a3 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:46.981328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:46.981946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:46.982224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:46.982496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:46.993346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:46.994789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:47.005860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1114/2205] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:32:47.285783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:32:47.287296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:32:47.300945 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b3659-8f96-48c5-be59-ee296b1fae09 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:32:47.305642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:32:47.306275 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:32:47.306554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:32:47.306823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] PATH_INFO: `/v3/projects/4d5dd788aafe4cd1841c71885bf824eb` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:32:47.307947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5dd788aafe4cd1841c71885bf824eb)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:32:47.310461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:32:47.357607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5dd788aafe4cd1841c71885bf824eb', 'request_id': 'req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:47.364362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5dd788aafe4cd1841c71885bf824eb', 'request_id': 'req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:47.372953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5dd788aafe4cd1841c71885bf824eb', 'request_id': 'req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:47.384217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] The token cache is being invalidate because project 4d5dd788aafe4cd1841c71885bf824eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:32:47.384506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:47.385132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d3d427b9-6a9d-4ee1-a1f0-036e7932f297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:32:47.386665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1092/2206] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:32:47 2022] DELETE /identity/v3/projects/4d5dd788aafe4cd1841c71885bf824eb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:32:57.001728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:32:57.381186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:04.810780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:04.813356 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:04.836626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78be45-56d2-402b-9f02-4c0f9c34b641 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:04.938781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:04.939435 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:04.939719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:04.939997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] PATH_INFO: `/v3/users/2e63e408731348858017070ddc558fd9` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:04.941432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e63e408731348858017070ddc558fd9)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:04.944721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:05.022608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e63e408731348858017070ddc558fd9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.040399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e63e408731348858017070ddc558fd9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.041563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e63e408731348858017070ddc558fd9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.043953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e63e408731348858017070ddc558fd9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.058949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e63e408731348858017070ddc558fd9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.085723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1115/2207] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:33:04 2022] DELETE /identity/v3/users/2e63e408731348858017070ddc558fd9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:05.090919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:05.091456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:05.092363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:05.120823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:05.120823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:05.213377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1093/2208] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:05.224446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:05.226315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:05.293775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f34f0-385a-4328-bf88-818715acb3d7 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:05.299090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f26cf6-14f9-4d75-b489-37baad806b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:05.299318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f26cf6-14f9-4d75-b489-37baad806b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:05.299973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f26cf6-14f9-4d75-b489-37baad806b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:05.299973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f26cf6-14f9-4d75-b489-37baad806b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:05.310313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f26cf6-14f9-4d75-b489-37baad806b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:05.311828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f26cf6-14f9-4d75-b489-37baad806b98 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:05.337604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1116/2209] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:05.667091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:05.668588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:05.681671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd099c-3523-4ea6-b6d0-1f62ad2e9aec None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:05.689626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:05.690498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:05.690782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:05.691060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] PATH_INFO: `/v3/projects/6e2a8e16ccc4426dbda0865ef0208bda` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:05.692480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2a8e16ccc4426dbda0865ef0208bda)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:05.694480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:05.744373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2a8e16ccc4426dbda0865ef0208bda', 'request_id': 'req-2973ace6-cf8f-4421-a247-581625b7f875'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.749988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2a8e16ccc4426dbda0865ef0208bda', 'request_id': 'req-2973ace6-cf8f-4421-a247-581625b7f875'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.757672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2a8e16ccc4426dbda0865ef0208bda', 'request_id': 'req-2973ace6-cf8f-4421-a247-581625b7f875'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.769789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] The token cache is being invalidate because project 6e2a8e16ccc4426dbda0865ef0208bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:33:05.770076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.770759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2973ace6-cf8f-4421-a247-581625b7f875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:05.772246 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1094/2210] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:33:05 2022] DELETE /identity/v3/projects/6e2a8e16ccc4426dbda0865ef0208bda => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:05.777919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d340a8-1fe2-4ac4-8e18-ac6009f69c9e None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:05.778201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d340a8-1fe2-4ac4-8e18-ac6009f69c9e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:05.778469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d340a8-1fe2-4ac4-8e18-ac6009f69c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:05.800450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a6d340a8-1fe2-4ac4-8e18-ac6009f69c9e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:05.801737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a6d340a8-1fe2-4ac4-8e18-ac6009f69c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:05.821042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:05.873568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1117/2211] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:05.881488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61ead-31c8-4453-8888-bb2634fcea63 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:05.881853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61ead-31c8-4453-8888-bb2634fcea63 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:05.882138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61ead-31c8-4453-8888-bb2634fcea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:05.902118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e1c61ead-31c8-4453-8888-bb2634fcea63 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:05.903337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e1c61ead-31c8-4453-8888-bb2634fcea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:05.911842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1095/2212] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:05.916191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d340a8-1fe2-4ac4-8e18-ac6009f69c9e None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:05.925744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d340a8-1fe2-4ac4-8e18-ac6009f69c9e None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:05.930758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:05.931707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:05.932104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:05.932415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:05.933094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:33:05.933442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:05.935452 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:05.948033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1118/2213] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:33:05.952879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61ead-31c8-4453-8888-bb2634fcea63 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:05.962223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61ead-31c8-4453-8888-bb2634fcea63 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:05.968615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:05.969220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:05.969530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:05.969802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:05.970436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:05.972662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:06.006559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1096/2214] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:33:05 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:33:06.007919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:06.019577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fde0d-3a16-4031-8777-c92c41bfa204 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:06.075678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:06.076226 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:06.076528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:06.077479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:06.078106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:06.080321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:06.108870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1119/2215] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:33:06 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:33:06.112841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:06.123653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f03be4-5180-40cb-897c-24e8b2560cf7 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:06.128724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:06.129361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:06.129640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:06.129910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:06.130581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:06.132162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:06.142265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1097/2216] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:06.146677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:06.165137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903eb7c-8bf5-43f9-974f-8612cd18b057 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:06.171054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf101b-4656-4be7-9574-5e1e9e43cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:06.172366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf101b-4656-4be7-9574-5e1e9e43cddb admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:06.172366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf101b-4656-4be7-9574-5e1e9e43cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:06.172366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf101b-4656-4be7-9574-5e1e9e43cddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:06.173006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf101b-4656-4be7-9574-5e1e9e43cddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:06.174665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf101b-4656-4be7-9574-5e1e9e43cddb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:06.186030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1120/2217] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:06.190135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:06.200571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88051655-de88-4bff-8dbb-6f2542d8f44d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:06.205554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:06.206209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:06.206498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:06.206776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] PATH_INFO: `/v3/projects/0b9ef5c23c3b47449690acb96883be58/users/266319f2026c4f7c865ccb5da10a7571/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:06.226930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9ef5c23c3b47449690acb96883be58, user_id=266319f2026c4f7c865ccb5da10a7571, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:06.226930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:06.242903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1098/2218] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:33:06 2022] PUT /identity/v3/projects/0b9ef5c23c3b47449690acb96883be58/users/266319f2026c4f7c865ccb5da10a7571/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:33:06.253222 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3c99-8cf6-43ce-bd89-1f7dd4db14b0 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:06.253539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3c99-8cf6-43ce-bd89-1f7dd4db14b0 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:06.253811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3c99-8cf6-43ce-bd89-1f7dd4db14b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:06.281489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a2de3c99-8cf6-43ce-bd89-1f7dd4db14b0 None None] MFA Rules not processed for user `266319f2026c4f7c865ccb5da10a7571`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:06.282757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a2de3c99-8cf6-43ce-bd89-1f7dd4db14b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:06.350233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1121/2219] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:06.363173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:06.364834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:06.401554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:06.431795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac5da4-8d66-45d0-b4b3-fcb16e2ca3b9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:06.435780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0f0bc-cd4c-4897-8776-e4b081c135fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:06.437034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0f0bc-cd4c-4897-8776-e4b081c135fe service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:06.437470 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0f0bc-cd4c-4897-8776-e4b081c135fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:06.437811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0f0bc-cd4c-4897-8776-e4b081c135fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:06.458574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0f0bc-cd4c-4897-8776-e4b081c135fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:06.460021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0f0bc-cd4c-4897-8776-e4b081c135fe service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:06.469074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1099/2220] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:11.122098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de3c99-8cf6-43ce-bd89-1f7dd4db14b0 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:11.141777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:11.190868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de3c99-8cf6-43ce-bd89-1f7dd4db14b0 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:11.197431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:11.198009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:11.198285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:11.198553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] PATH_INFO: `/v3/users/266319f2026c4f7c865ccb5da10a7571` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:11.199697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=266319f2026c4f7c865ccb5da10a7571)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:11.201700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:11.247407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266319f2026c4f7c865ccb5da10a7571'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.254810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266319f2026c4f7c865ccb5da10a7571'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.255579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266319f2026c4f7c865ccb5da10a7571'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.256184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266319f2026c4f7c865ccb5da10a7571'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.267031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266319f2026c4f7c865ccb5da10a7571'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.284606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1122/2221] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:33:11 2022] DELETE /identity/v3/users/266319f2026c4f7c865ccb5da10a7571 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:11.289013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:11.289481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:11.289585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:11.309080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:11.310310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:11.372226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1100/2222] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:11.382065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:11.383487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:11.442977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cb616-8dc5-4580-9b58-aed41421a39d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:11.446774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607dae3-657a-43dc-ad34-9eb5f4939137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:11.447427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607dae3-657a-43dc-ad34-9eb5f4939137 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:11.447755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607dae3-657a-43dc-ad34-9eb5f4939137 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:11.447974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607dae3-657a-43dc-ad34-9eb5f4939137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:11.457642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607dae3-657a-43dc-ad34-9eb5f4939137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:11.459070 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607dae3-657a-43dc-ad34-9eb5f4939137 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:11.469151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1123/2223] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:11.750808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:11.752342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:11.772230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb8cc0-9f1e-4c36-8ec0-d23ef9ddd1e2 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:11.788453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:11.789187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:11.789500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:11.789780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] PATH_INFO: `/v3/projects/0b9ef5c23c3b47449690acb96883be58` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:11.792004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9ef5c23c3b47449690acb96883be58)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:11.794628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:11.860386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9ef5c23c3b47449690acb96883be58', 'request_id': 'req-1f815fb3-ab48-469d-9fab-9a59aaec5171'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.866074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9ef5c23c3b47449690acb96883be58', 'request_id': 'req-1f815fb3-ab48-469d-9fab-9a59aaec5171'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.873727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9ef5c23c3b47449690acb96883be58', 'request_id': 'req-1f815fb3-ab48-469d-9fab-9a59aaec5171'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.890138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] The token cache is being invalidate because project 0b9ef5c23c3b47449690acb96883be58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:33:11.890432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.891066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1f815fb3-ab48-469d-9fab-9a59aaec5171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:11.892549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1101/2224] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:33:11 2022] DELETE /identity/v3/projects/0b9ef5c23c3b47449690acb96883be58 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:33:11.897430 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77faf9-8b15-49a9-958b-b96d104d1cd4 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:11.897713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77faf9-8b15-49a9-958b-b96d104d1cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:11.897982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77faf9-8b15-49a9-958b-b96d104d1cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:11.917628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8c77faf9-8b15-49a9-958b-b96d104d1cd4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:11.918807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8c77faf9-8b15-49a9-958b-b96d104d1cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:11.953620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:11.983666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1124/2225] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:11.991442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d82e-47d9-44ec-a147-1de36dc54413 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:11.991736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d82e-47d9-44ec-a147-1de36dc54413 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:11.992012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390d82e-47d9-44ec-a147-1de36dc54413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.011870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-c390d82e-47d9-44ec-a147-1de36dc54413 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:12.013070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c390d82e-47d9-44ec-a147-1de36dc54413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:12.021574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1102/2226] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:12.025669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77faf9-8b15-49a9-958b-b96d104d1cd4 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:12.035130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77faf9-8b15-49a9-958b-b96d104d1cd4 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:12.039677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:12.040270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.040545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.040815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.041562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:33:12.042001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:12.044256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:12.055854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1125/2227] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:33:12.060633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390d82e-47d9-44ec-a147-1de36dc54413 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:12.070394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390d82e-47d9-44ec-a147-1de36dc54413 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:12.075277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:12.075898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.076172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.076455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.077089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:12.079118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:12.107391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1103/2228] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:33:12 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:33:12.111649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:12.122097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba1bf9-f65c-4973-98ec-8996a445c0e3 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:12.179723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:12.180759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.181055 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.181511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.182136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:12.184249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:12.208561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1126/2229] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:33:12 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:33:12.212725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:12.223532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9433e-bc1f-467b-8330-dedfec299a0d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:12.228804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:12.230103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.230430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.230707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.231458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:12.233720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:12.244797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1104/2230] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:12.250065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:12.260619 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202cd43-7738-44bd-8c46-557f8de01a8c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:12.265866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49ce77-1389-436c-a2ed-3bd6166eb7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:12.266466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49ce77-1389-436c-a2ed-3bd6166eb7af admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.266739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49ce77-1389-436c-a2ed-3bd6166eb7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.267006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49ce77-1389-436c-a2ed-3bd6166eb7af admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.267642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49ce77-1389-436c-a2ed-3bd6166eb7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:12.269239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49ce77-1389-436c-a2ed-3bd6166eb7af admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:12.280465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1127/2231] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:12.284488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:12.295317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3046a-2d0c-4a3d-85c8-c01b7dbfdf34 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:12.301373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:12.302106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.302880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.303332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] PATH_INFO: `/v3/projects/f67d0c19d47c4440a3eac88c79a61639/users/3994f571a8b145fb929bc0e2c1fcdb49/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.306064 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:12.322407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67d0c19d47c4440a3eac88c79a61639, user_id=3994f571a8b145fb929bc0e2c1fcdb49, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:12.325190 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:12.338695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1105/2232] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:33:12 2022] PUT /identity/v3/projects/f67d0c19d47c4440a3eac88c79a61639/users/3994f571a8b145fb929bc0e2c1fcdb49/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:12.350649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.350932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.351199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.380467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] MFA Rules not processed for user `3994f571a8b145fb929bc0e2c1fcdb49`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:12.381777 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:12.444925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1128/2233] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:12.455640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:12.457389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:12.520003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd146-ad42-447b-b271-d070af075b7b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:12.523599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0d4de-b293-4b0d-8f32-e9908a545938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:12.524195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0d4de-b293-4b0d-8f32-e9908a545938 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:12.524469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0d4de-b293-4b0d-8f32-e9908a545938 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:12.524737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0d4de-b293-4b0d-8f32-e9908a545938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:12.532716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0d4de-b293-4b0d-8f32-e9908a545938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:12.534182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0d4de-b293-4b0d-8f32-e9908a545938 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:12.542909 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1106/2234] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:14.011495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:14.013157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:14.058879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:14.084905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad68b5d-c6e4-468e-a19b-8cc2ea41b3be None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:14.089796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:14.090593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:14.090907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:14.091185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] PATH_INFO: `/v3/users/563fb73ed5ef408db830abadb69ebc27` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:14.092555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563fb73ed5ef408db830abadb69ebc27)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:14.094571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:14.151693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563fb73ed5ef408db830abadb69ebc27'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.164281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563fb73ed5ef408db830abadb69ebc27'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.165166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563fb73ed5ef408db830abadb69ebc27'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.166118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563fb73ed5ef408db830abadb69ebc27'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.175590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563fb73ed5ef408db830abadb69ebc27'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.202259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1129/2235] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:33:14 2022] DELETE /identity/v3/users/563fb73ed5ef408db830abadb69ebc27 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:14.207387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:14.207927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:14.207927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:14.230295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:14.231836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:14.322303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1107/2236] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:14.332450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:14.333965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:14.391526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5a67f-548c-4d05-8eaf-95e7cd42d121 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:14.395188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb680ab-ee86-4390-baf5-d1c8bbff6d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:14.395790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb680ab-ee86-4390-baf5-d1c8bbff6d4a service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:14.396073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb680ab-ee86-4390-baf5-d1c8bbff6d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:14.396351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb680ab-ee86-4390-baf5-d1c8bbff6d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:14.405259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb680ab-ee86-4390-baf5-d1c8bbff6d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:14.406719 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb680ab-ee86-4390-baf5-d1c8bbff6d4a service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:14.416071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1130/2237] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:33:14.791355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:14.792969 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:14.807771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271a032-d01f-42d6-b411-edcdde65f1c3 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:14.813050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:14.813728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:14.814016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:14.814296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] PATH_INFO: `/v3/projects/27cb93320e3f4e5ba64ce9541c405336` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:14.815451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cb93320e3f4e5ba64ce9541c405336)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:14.817428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:14.864646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cb93320e3f4e5ba64ce9541c405336', 'request_id': 'req-d0d5bd1f-b444-4780-a522-905e8b5cbde1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.870767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cb93320e3f4e5ba64ce9541c405336', 'request_id': 'req-d0d5bd1f-b444-4780-a522-905e8b5cbde1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.878724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cb93320e3f4e5ba64ce9541c405336', 'request_id': 'req-d0d5bd1f-b444-4780-a522-905e8b5cbde1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.889779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] The token cache is being invalidate because project 27cb93320e3f4e5ba64ce9541c405336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:33:14.890077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.890705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d0d5bd1f-b444-4780-a522-905e8b5cbde1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:14.892286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1108/2238] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:33:14 2022] DELETE /identity/v3/projects/27cb93320e3f4e5ba64ce9541c405336 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:14.897605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdab2a-b06f-4956-84ed-86d1e86bb7c5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:14.897895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdab2a-b06f-4956-84ed-86d1e86bb7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:14.898271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdab2a-b06f-4956-84ed-86d1e86bb7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:14.917152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-57fdab2a-b06f-4956-84ed-86d1e86bb7c5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:14.918453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-57fdab2a-b06f-4956-84ed-86d1e86bb7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:14.968600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:14.979671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1131/2239] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:14.988768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336a120-30b4-4c81-9356-2ef278c34022 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:14.989131 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336a120-30b4-4c81-9356-2ef278c34022 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:14.989508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336a120-30b4-4c81-9356-2ef278c34022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.009309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6336a120-30b4-4c81-9356-2ef278c34022 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:15.010518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6336a120-30b4-4c81-9356-2ef278c34022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:15.018400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1109/2240] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:15.022429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdab2a-b06f-4956-84ed-86d1e86bb7c5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:15.032483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdab2a-b06f-4956-84ed-86d1e86bb7c5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:15.037113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:15.037754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.038028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.038296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.038951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:33:15.039254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:15.040861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:15.054779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1132/2241] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:33:15.057771 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336a120-30b4-4c81-9356-2ef278c34022 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:15.062262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:15.068476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336a120-30b4-4c81-9356-2ef278c34022 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:15.074445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:15.075050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.075335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.075614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.076243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:15.078276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:15.105957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1110/2242] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:33:15 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:33:15.110779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:15.120781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff5c22-0981-4b0b-8bc2-16574f694c87 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:15.176729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:15.177371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.177657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.177933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.178568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:15.180512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:15.200745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1133/2243] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:33:15 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:33:15.204766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:15.214908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043b198-a674-41d5-af88-686fb1c70ebf admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:15.220163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:15.220767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.221047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.221350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.222259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:15.223922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:15.234341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1111/2244] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:15.237970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:15.248601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d5609-9807-4a22-8d2e-6a1675282281 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:15.253284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb6509-fa6a-4ff9-970f-e55e5a59dff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:15.253878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6509-fa6a-4ff9-970f-e55e5a59dff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.254161 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6509-fa6a-4ff9-970f-e55e5a59dff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.254436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6509-fa6a-4ff9-970f-e55e5a59dff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.255076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb6509-fa6a-4ff9-970f-e55e5a59dff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:15.256950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb6509-fa6a-4ff9-970f-e55e5a59dff0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:15.267441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1134/2245] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:15.271428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:15.281610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6d28-1feb-4a8c-af25-4826f77ce31a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:15.286157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:15.286794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.287072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.287349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] PATH_INFO: `/v3/projects/cb4fb7d34a90445e9677ce7287759724/users/618d31fc46f64971af696b219e8848a3/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.302446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4fb7d34a90445e9677ce7287759724, user_id=618d31fc46f64971af696b219e8848a3, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:15.305184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:15.319322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1112/2246] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:33:15 2022] PUT /identity/v3/projects/cb4fb7d34a90445e9677ce7287759724/users/618d31fc46f64971af696b219e8848a3/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:15.324868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da874a6e-f554-48b1-9892-54389c70f900 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.325210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da874a6e-f554-48b1-9892-54389c70f900 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.325740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da874a6e-f554-48b1-9892-54389c70f900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.347856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-da874a6e-f554-48b1-9892-54389c70f900 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:15.349109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-da874a6e-f554-48b1-9892-54389c70f900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:15.417742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1135/2247] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:15.430297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:15.432124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:15.496599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facf73d-0a40-4bf6-a7eb-045d3b90685f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:15.501173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa5fc4-fe06-4104-b892-61ea269c256c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:15.501887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5fc4-fe06-4104-b892-61ea269c256c service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:15.502363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5fc4-fe06-4104-b892-61ea269c256c service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:15.502671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5fc4-fe06-4104-b892-61ea269c256c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:15.512614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa5fc4-fe06-4104-b892-61ea269c256c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:15.514840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa5fc4-fe06-4104-b892-61ea269c256c service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:15.528110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1113/2248] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:19.512724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f66f-d29e-4035-bc5e-8a53783e2778 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:19.513219 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f66f-d29e-4035-bc5e-8a53783e2778 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:19.513332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f66f-d29e-4035-bc5e-8a53783e2778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:19.549025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9fd0f66f-d29e-4035-bc5e-8a53783e2778 None None] MFA Rules not processed for user `9b8e4aad09814f8299a334639e34d948`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:19.549025 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9fd0f66f-d29e-4035-bc5e-8a53783e2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:19.696711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1136/2249] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:21.714196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a7dda-e569-456f-beb4-eba5cef3f137 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:21.714754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a7dda-e569-456f-beb4-eba5cef3f137 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:21.714877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a7dda-e569-456f-beb4-eba5cef3f137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:21.746118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ee2a7dda-e569-456f-beb4-eba5cef3f137 None None] MFA Rules not processed for user `618d31fc46f64971af696b219e8848a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:21.747446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ee2a7dda-e569-456f-beb4-eba5cef3f137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:21.985926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1114/2250] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:21.990699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f66f-d29e-4035-bc5e-8a53783e2778 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:22.017665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f66f-d29e-4035-bc5e-8a53783e2778 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:22.026505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:22.027587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:22.028011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:22.028495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:22.029592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:22.031642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:22.038145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:22.074243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1137/2251] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:33:21 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:33:22.075418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a7dda-e569-456f-beb4-eba5cef3f137 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:22.108129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a7dda-e569-456f-beb4-eba5cef3f137 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:22.150625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:22.175490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:22.176219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:22.176566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:22.176932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:22.177683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:22.179669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:22.202450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1115/2252] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:33:22 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:33:22.206417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:22.216612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44659030-62e9-40e5-a77c-ba9b2b260a76 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:22.221472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:22.222429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:22.222776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:22.223238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:22.223939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:22.225588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:22.240071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1138/2253] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:22.241435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:22.252262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f35a-a6ca-4cdb-bc68-0f6d152846cb admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:22.256914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:22.257622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:22.257898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:22.258167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] PATH_INFO: `/v3/projects/a0797133ed164db5bee55fe1578d2aee/users/f959af6a61ae46fd8d9e00904194625f/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:22.274369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0797133ed164db5bee55fe1578d2aee, user_id=f959af6a61ae46fd8d9e00904194625f, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:22.277049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:22.291331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1116/2254] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:33:22 2022] PUT /identity/v3/projects/a0797133ed164db5bee55fe1578d2aee/users/f959af6a61ae46fd8d9e00904194625f/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:33:22.298419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:22.308860 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edb4c1-9494-43bf-9d66-d8e54b453ee0 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:22.368511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:22.369046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:22.369362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:22.369643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:22.370288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:22.371893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:22.382587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1139/2255] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:33:22.386624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:22.397182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8161e-7c0d-4b89-b176-fc2f59cad656 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:22.401796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705a515-a24d-4887-957b-7f830dac0275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:22.402450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705a515-a24d-4887-957b-7f830dac0275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:22.402730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705a515-a24d-4887-957b-7f830dac0275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:22.402999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705a515-a24d-4887-957b-7f830dac0275 admin admin] PATH_INFO: `/v3/projects/a0797133ed164db5bee55fe1578d2aee/users/f959af6a61ae46fd8d9e00904194625f/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:22.405516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705a515-a24d-4887-957b-7f830dac0275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0797133ed164db5bee55fe1578d2aee, user_id=f959af6a61ae46fd8d9e00904194625f, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:22.408185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705a515-a24d-4887-957b-7f830dac0275 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:22.419288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1117/2256] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:33:22 2022] PUT /identity/v3/projects/a0797133ed164db5bee55fe1578d2aee/users/f959af6a61ae46fd8d9e00904194625f/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:23.912176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:23.914394 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:23.950522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1a2fa-bfa2-4d81-ab1d-29295d0d2aec admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:24.094905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:24.095911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:24.096399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:24.097321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] PATH_INFO: `/v3/users/3994f571a8b145fb929bc0e2c1fcdb49` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:24.098743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3994f571a8b145fb929bc0e2c1fcdb49)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:24.101841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:24.229195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3994f571a8b145fb929bc0e2c1fcdb49'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:24.246190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3994f571a8b145fb929bc0e2c1fcdb49'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:24.246955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3994f571a8b145fb929bc0e2c1fcdb49'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:24.248495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3994f571a8b145fb929bc0e2c1fcdb49'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:24.259749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3994f571a8b145fb929bc0e2c1fcdb49'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:24.281785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:24.308735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1140/2257] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:33:23 2022] DELETE /identity/v3/users/3994f571a8b145fb929bc0e2c1fcdb49 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:24.314915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:24.315794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:24.315794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:24.354390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:24.368121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:24.537743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1118/2258] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:24.567521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:24.569019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:24.691203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1860ad-973d-4d36-8749-9db3ada4a7b2 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:24.696292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ce662-3b80-4ca6-9b30-e519ee182f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:24.697027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ce662-3b80-4ca6-9b30-e519ee182f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:24.697457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ce662-3b80-4ca6-9b30-e519ee182f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:24.697939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ce662-3b80-4ca6-9b30-e519ee182f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:24.724987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ce662-3b80-4ca6-9b30-e519ee182f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:24.727276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ce662-3b80-4ca6-9b30-e519ee182f7d service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:24.741027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1141/2259] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:25.068978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:25.070573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:25.097194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734835ac-d25f-4f41-8e1a-87fc062073ff None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:25.125770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:25.126734 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:25.127214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:25.127688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] PATH_INFO: `/v3/users/52e0a145380148a1bbeb3562991843e8` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:25.129069 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e0a145380148a1bbeb3562991843e8)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:25.131386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:25.232862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e0a145380148a1bbeb3562991843e8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:25.246727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e0a145380148a1bbeb3562991843e8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:25.277736 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e0a145380148a1bbeb3562991843e8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:25.278767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e0a145380148a1bbeb3562991843e8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:25.280184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e0a145380148a1bbeb3562991843e8'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:25.319254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1119/2260] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:33:25 2022] DELETE /identity/v3/users/52e0a145380148a1bbeb3562991843e8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:25.324081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:25.324577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:25.324655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:25.356709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:25.358850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:25.514642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1142/2261] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:25.558124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:25.559812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:25.587623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:25.751398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9feb2-72a2-4f24-bd46-67bb972acb78 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:25.756672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:25.758023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:25.758023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:25.758023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:25.786279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:25.787904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:25.812061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1120/2262] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:26.349543 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:26.352349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:26.379530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ff38a-9f35-46cd-8f75-38da109da575 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:26.398157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:26.414474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:26.416425 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:26.416947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] PATH_INFO: `/v3/projects/f67d0c19d47c4440a3eac88c79a61639` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:26.420727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67d0c19d47c4440a3eac88c79a61639)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:26.424304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:26.570859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67d0c19d47c4440a3eac88c79a61639', 'request_id': 'req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:26.589725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67d0c19d47c4440a3eac88c79a61639', 'request_id': 'req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:26.608574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67d0c19d47c4440a3eac88c79a61639', 'request_id': 'req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:26.631890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] The token cache is being invalidate because project f67d0c19d47c4440a3eac88c79a61639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:33:26.632192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:26.632899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e4982de-6b6e-411f-afa0-6602d6f4c9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:26.635067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1143/2263] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:33:26 2022] DELETE /identity/v3/projects/f67d0c19d47c4440a3eac88c79a61639 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:26.777606 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:26.779359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:26.864744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0df68-beb9-45ed-9828-4a539d72ba6d service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:26.972145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:26.973188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:26.973725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:26.974192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] PATH_INFO: `/v3/projects/4cae2733cd51406b9e572cc98b091eac` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:26.975836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cae2733cd51406b9e572cc98b091eac)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:26.979187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:27.070564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cae2733cd51406b9e572cc98b091eac', 'request_id': 'req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:27.084005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cae2733cd51406b9e572cc98b091eac', 'request_id': 'req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:27.105429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cae2733cd51406b9e572cc98b091eac', 'request_id': 'req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:27.133883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] The token cache is being invalidate because project 4cae2733cd51406b9e572cc98b091eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:33:27.134502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:27.135327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:33:27.137400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1121/2264] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:33:26 2022] DELETE /identity/v3/projects/4cae2733cd51406b9e572cc98b091eac => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:33:28.356132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb7336-b485-47a1-9dc8-ed266b52165a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:28.357109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb7336-b485-47a1-9dc8-ed266b52165a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:28.357786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb7336-b485-47a1-9dc8-ed266b52165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:28.385596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-40cb7336-b485-47a1-9dc8-ed266b52165a None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:28.387220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-40cb7336-b485-47a1-9dc8-ed266b52165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:28.422732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:28.525648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1144/2265] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:28.562290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:28.564240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:28.648423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:28.707380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83456-e3de-4b5e-afd6-6d10f58ae2b2 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:28.714158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:28.715513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:28.715975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:28.716454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:28.728234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:28.731421 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:28.743806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1122/2266] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:33:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:33:28.985680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d244eb-eef3-4e1d-a791-d8c27d7f487f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:28.985972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d244eb-eef3-4e1d-a791-d8c27d7f487f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:28.986246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d244eb-eef3-4e1d-a791-d8c27d7f487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:29.021701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-74d244eb-eef3-4e1d-a791-d8c27d7f487f None None] MFA Rules not processed for user `f959af6a61ae46fd8d9e00904194625f`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:29.022931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-74d244eb-eef3-4e1d-a791-d8c27d7f487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:29.121315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1145/2267] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:29.140879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:29.142578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:29.246144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21f594-fd15-4c89-b9d5-986e7acfd106 service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:29.263855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9be8c8c5f7f6417e9ab19ef5bce0f060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:29.264644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:29.265032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:29.265445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:29.271022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:29.358944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:29.360743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:29.371831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1123/2268] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:29.578304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b12dd3-7b59-4f3d-bd0e-2ea6658e31a6 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:29.582756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b12dd3-7b59-4f3d-bd0e-2ea6658e31a6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:29.583186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b12dd3-7b59-4f3d-bd0e-2ea6658e31a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:29.608569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-81b12dd3-7b59-4f3d-bd0e-2ea6658e31a6 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:29.609971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-81b12dd3-7b59-4f3d-bd0e-2ea6658e31a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:29.626000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1146/2269] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:33:29.636006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:29.651746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e41c5-9e0d-47a9-a453-ba9683f88b4f service glance] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:29.657900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f28c2-75eb-4913-987f-bcd5b1b66618 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:29.659994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f28c2-75eb-4913-987f-bcd5b1b66618 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:29.659994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f28c2-75eb-4913-987f-bcd5b1b66618 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:29.659994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f28c2-75eb-4913-987f-bcd5b1b66618 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:29.694177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f28c2-75eb-4913-987f-bcd5b1b66618 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:29.699544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f28c2-75eb-4913-987f-bcd5b1b66618 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:29.725845 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1124/2270] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:33:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:33:37.298824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12a8fc-5f3f-43cd-98e7-1a834cc1d0dd None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:37.300461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12a8fc-5f3f-43cd-98e7-1a834cc1d0dd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:37.300966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12a8fc-5f3f-43cd-98e7-1a834cc1d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:37.328801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5b12a8fc-5f3f-43cd-98e7-1a834cc1d0dd None None] MFA Rules not processed for user `2aa6d6d56f9f4618ae3883c9beb2e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:37.332805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5b12a8fc-5f3f-43cd-98e7-1a834cc1d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:37.462707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1147/2271] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:39.714703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=70 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:43.513618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1ec8-d919-4e3e-897d-fc224aa987aa None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:43.514252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1ec8-d919-4e3e-897d-fc224aa987aa None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:43.514252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1ec8-d919-4e3e-897d-fc224aa987aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:43.533590 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8a2e1ec8-d919-4e3e-897d-fc224aa987aa None None] MFA Rules not processed for user `618d31fc46f64971af696b219e8848a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:43.535093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8a2e1ec8-d919-4e3e-897d-fc224aa987aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:43.542393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1125/2272] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:47.421901 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=79 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:48.891320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe8ee6-ce28-487a-9403-31a88ded7393 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:48.892403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe8ee6-ce28-487a-9403-31a88ded7393 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:48.892969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe8ee6-ce28-487a-9403-31a88ded7393 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:48.894900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1148/2273] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:33:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:48.901003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:48.901518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:48.901623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:48.920670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:48.922152 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:49.018670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1126/2274] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:49.022869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be4c2-cb39-47c4-978d-9aef0bf13fde None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.023482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be4c2-cb39-47c4-978d-9aef0bf13fde None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.023991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be4c2-cb39-47c4-978d-9aef0bf13fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.031579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:49.033213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:49.046074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ef4be4c2-cb39-47c4-978d-9aef0bf13fde None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:49.047462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ef4be4c2-cb39-47c4-978d-9aef0bf13fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:49.051011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0be4-7817-4d1d-92a6-4f4d20cbe1db None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:49.054872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1149/2275] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:49.055294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909033b-7938-4872-a208-632cfd1e02b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9be8c8c5f7f6417e9ab19ef5bce0f060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:49.055941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909033b-7938-4872-a208-632cfd1e02b6 service glance] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.056229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909033b-7938-4872-a208-632cfd1e02b6 service glance] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.056505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909033b-7938-4872-a208-632cfd1e02b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.068724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4be4c2-cb39-47c4-978d-9aef0bf13fde None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:49.069361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909033b-7938-4872-a208-632cfd1e02b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:49.071518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909033b-7938-4872-a208-632cfd1e02b6 service glance] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:49.082058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4be4c2-cb39-47c4-978d-9aef0bf13fde None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:49.085862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1127/2276] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:33:49.087891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b3c62-3d43-4d61-95f2-81085058da1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:49.088569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3c62-3d43-4d61-95f2-81085058da1e service swift] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.089049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3c62-3d43-4d61-95f2-81085058da1e service swift] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.089436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3c62-3d43-4d61-95f2-81085058da1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.100422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b3c62-3d43-4d61-95f2-81085058da1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:49.102512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b3c62-3d43-4d61-95f2-81085058da1e service swift] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:49.129587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1150/2277] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:33:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 12:33:49.177875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a10af-2bd5-40ff-a4f4-0d6785b948a6 None None] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.178163 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a10af-2bd5-40ff-a4f4-0d6785b948a6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.178444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a10af-2bd5-40ff-a4f4-0d6785b948a6 None None] PATH_INFO: `/` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.180342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1128/2278] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:49.185447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e826a-4cb8-4e07-92ca-e31357bc8f60 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.185734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e826a-4cb8-4e07-92ca-e31357bc8f60 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.186008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e826a-4cb8-4e07-92ca-e31357bc8f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.207008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-df7e826a-4cb8-4e07-92ca-e31357bc8f60 None None] MFA Rules not processed for user `8c4588df9de242e8819de0476f75959c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:49.208622 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-df7e826a-4cb8-4e07-92ca-e31357bc8f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:49.330256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1151/2279] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:49.346466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a10af-2bd5-40ff-a4f4-0d6785b948a6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:49.348562 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-973a10af-2bd5-40ff-a4f4-0d6785b948a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:49.382188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a10af-2bd5-40ff-a4f4-0d6785b948a6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:49.396410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116c99e-f33f-430c-9a00-aff7477b4e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:49.399025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116c99e-f33f-430c-9a00-aff7477b4e5c service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.399918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116c99e-f33f-430c-9a00-aff7477b4e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.401964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116c99e-f33f-430c-9a00-aff7477b4e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.428922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116c99e-f33f-430c-9a00-aff7477b4e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:49.430911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116c99e-f33f-430c-9a00-aff7477b4e5c service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:49.461922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1129/2280] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:49.643876 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994570f-b833-46a5-af06-78f9ca1b8b64 None None] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.644321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994570f-b833-46a5-af06-78f9ca1b8b64 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.644694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994570f-b833-46a5-af06-78f9ca1b8b64 None None] PATH_INFO: `/` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.646168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1152/2281] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 10 12:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 12:33:49.671822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6570d4e-200c-44ff-a9d9-5fa7b45d16f6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.672493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6570d4e-200c-44ff-a9d9-5fa7b45d16f6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.672971 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6570d4e-200c-44ff-a9d9-5fa7b45d16f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.701462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-c6570d4e-200c-44ff-a9d9-5fa7b45d16f6 None None] MFA Rules not processed for user `536ca83c26354a77b1c5d263c57f4f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:33:49.702997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c6570d4e-200c-44ff-a9d9-5fa7b45d16f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:33:49.712521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1130/2282] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 10 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:33:49.725223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994570f-b833-46a5-af06-78f9ca1b8b64 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:33:49.739565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994570f-b833-46a5-af06-78f9ca1b8b64 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:33:49.755822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:33:49.756662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:33:49.757058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:33:49.757464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:33:49.773383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:33:49.774993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:33:49.788105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1153/2283] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:33:59.695911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=49 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:33:59.782212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=38 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:07.362958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:07.363454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:07.363544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:07.383892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:07.385030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:07.392627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1131/2284] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:07.404899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:07.416479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0a28-2c7c-47c0-b52b-41cc2d2232aa service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:07.421401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbdf84-c3fc-4b06-9650-846ec69fbd29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:07.421989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbdf84-c3fc-4b06-9650-846ec69fbd29 service swift] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:07.422267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbdf84-c3fc-4b06-9650-846ec69fbd29 service swift] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:07.422571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbdf84-c3fc-4b06-9650-846ec69fbd29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:07.430815 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbdf84-c3fc-4b06-9650-846ec69fbd29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:07.432400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbdf84-c3fc-4b06-9650-846ec69fbd29 service swift] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:07.440872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1154/2285] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:34:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:34:17.378950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:17.437645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:23.117644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:23.119784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:23.224832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4411f-b088-482e-9f6b-6cca32d94c90 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:23.231375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:23.232428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:23.232889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:23.233918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:23.242204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:23.262709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:23.264940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:23.285704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1132/2286] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:32.810397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:32.810900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:32.811001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:32.831577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:32.832786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:32.839953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1155/2287] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 10 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:34:32.862616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:32.873691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab34744-99e1-4252-a0c6-3849fd14d443 service placement] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:32.878666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd013f-3577-484c-8af0-4dfb68a4180b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:32.879254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd013f-3577-484c-8af0-4dfb68a4180b service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:32.879538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd013f-3577-484c-8af0-4dfb68a4180b service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:32.879809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd013f-3577-484c-8af0-4dfb68a4180b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:32.887611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd013f-3577-484c-8af0-4dfb68a4180b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:32.889773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd013f-3577-484c-8af0-4dfb68a4180b service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:32.897623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1133/2288] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:34:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:34:35.473635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:34:35.475132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:35.486417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5384b4-39d4-4aad-95d0-6fce66a3586d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:34:35.490014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeece70-5966-4de9-9745-fcb791329292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:34:35.490716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeece70-5966-4de9-9745-fcb791329292 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:35.491062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeece70-5966-4de9-9745-fcb791329292 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:35.491338 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeece70-5966-4de9-9745-fcb791329292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:35.500655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeece70-5966-4de9-9745-fcb791329292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:34:35.502110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeece70-5966-4de9-9745-fcb791329292 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:34:35.512043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1156/2289] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:34:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:34:42.894564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=38 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:45.507508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=9 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:34:52.653847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb8b30-a798-4c67-8321-4ad78f83c0ac None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:34:52.654337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb8b30-a798-4c67-8321-4ad78f83c0ac None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:34:52.654440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb8b30-a798-4c67-8321-4ad78f83c0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:34:52.673869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8ebb8b30-a798-4c67-8321-4ad78f83c0ac None None] MFA Rules not processed for user `9b8e4aad09814f8299a334639e34d948`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:34:52.675060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8ebb8b30-a798-4c67-8321-4ad78f83c0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:34:52.728641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1134/2290] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:34:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:02.722045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=25 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:34.448626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9dbf5-d2ae-482f-a0ca-ffb9fffdf9dc None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:34.449149 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9dbf5-d2ae-482f-a0ca-ffb9fffdf9dc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:34.449249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9dbf5-d2ae-482f-a0ca-ffb9fffdf9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:34.468402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-84f9dbf5-d2ae-482f-a0ca-ffb9fffdf9dc None None] MFA Rules not processed for user `a78bd1aed509474b965592c478cb1c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:34.469632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-84f9dbf5-d2ae-482f-a0ca-ffb9fffdf9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:34.526527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1157/2291] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:35:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:35:44.519900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=25 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:35:51.303023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617079d1-3c23-4690-8f2f-816113186174 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:35:51.303569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617079d1-3c23-4690-8f2f-816113186174 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:35:51.303672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617079d1-3c23-4690-8f2f-816113186174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:35:51.322910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-617079d1-3c23-4690-8f2f-816113186174 None None] MFA Rules not processed for user `2aa6d6d56f9f4618ae3883c9beb2e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:35:51.324101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-617079d1-3c23-4690-8f2f-816113186174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:35:51.331493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1135/2292] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:01.318389 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:04.939329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9dbf5-d2ae-482f-a0ca-ffb9fffdf9dc None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:04.948879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9dbf5-d2ae-482f-a0ca-ffb9fffdf9dc None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:04.952482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0887c16-69c0-4ca9-ac3d-7de03bad34f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:04.953075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0887c16-69c0-4ca9-ac3d-7de03bad34f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:04.953392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0887c16-69c0-4ca9-ac3d-7de03bad34f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:04.953670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0887c16-69c0-4ca9-ac3d-7de03bad34f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:04.955880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b0887c16-69c0-4ca9-ac3d-7de03bad34f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:04.968776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0887c16-69c0-4ca9-ac3d-7de03bad34f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:04.970885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0887c16-69c0-4ca9-ac3d-7de03bad34f0 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:04.981791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1158/2293] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:06.159531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617079d1-3c23-4690-8f2f-816113186174 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:06.169177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617079d1-3c23-4690-8f2f-816113186174 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:06.173379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:06.173972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:06.174248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:06.174518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:06.176232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:06.189020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:06.190459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:06.201029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1136/2294] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:10.661101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4a1d-6879-4788-9c41-da74e7b731ce None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:10.661618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4a1d-6879-4788-9c41-da74e7b731ce None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:10.661725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4a1d-6879-4788-9c41-da74e7b731ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:10.680948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a4ab4a1d-6879-4788-9c41-da74e7b731ce None None] MFA Rules not processed for user `9b8e4aad09814f8299a334639e34d948`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:10.682192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a4ab4a1d-6879-4788-9c41-da74e7b731ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:10.688934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1159/2295] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:36:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:36:15.978071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:15.979591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:15.993853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bdbab-8fdb-4e4b-94d7-1b26e4e894da service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:15.999345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:15.999938 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:16.000213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:16.001150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:16.002892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:16.013449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:16.015027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:16.024032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1137/2296] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:36:20.676239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=9 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:25.253470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca91bb-5ffe-4786-a82b-115b5254e52c None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:25.254067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca91bb-5ffe-4786-a82b-115b5254e52c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:25.254067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca91bb-5ffe-4786-a82b-115b5254e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:25.274559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-46ca91bb-5ffe-4786-a82b-115b5254e52c None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:25.276131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-46ca91bb-5ffe-4786-a82b-115b5254e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:25.285489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1160/2297] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 10 12:36:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:25.342171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:25.354719 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c788f0a-5e05-45d2-8041-59c9ceb66ad4 service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:25.360122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:25.360978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:25.361261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:25.361563 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:25.370324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:25.371907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:25.381904 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1138/2298] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:36:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:36:31.969448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca91bb-5ffe-4786-a82b-115b5254e52c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:31.984492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca91bb-5ffe-4786-a82b-115b5254e52c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:31.990315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:31.991033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:31.991367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:31.991850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:31.993900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:32.006653 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:32.008386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:32.021059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1161/2299] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:32.040897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:32.042381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:32.048652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=19 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:32.124253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63298d5c-827e-49cb-8a63-1b63307c3f14 service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:32.128763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:32.129467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:32.129750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:32.130025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] PATH_INFO: `/v3/users/618d31fc46f64971af696b219e8848a3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:32.131178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=618d31fc46f64971af696b219e8848a3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:32.133106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:32.180188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618d31fc46f64971af696b219e8848a3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.186867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618d31fc46f64971af696b219e8848a3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.194147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618d31fc46f64971af696b219e8848a3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.194892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618d31fc46f64971af696b219e8848a3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.195601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618d31fc46f64971af696b219e8848a3'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.214699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1139/2300] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:36:32 2022] DELETE /identity/v3/users/618d31fc46f64971af696b219e8848a3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:36:32.387422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:32.401015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89348a18-eb7a-44e0-941f-5a36e999fe57 service placement] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:32.463996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:32.464720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:32.465093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:32.465482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] PATH_INFO: `/v3/projects/cb4fb7d34a90445e9677ce7287759724` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:32.466672 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4fb7d34a90445e9677ce7287759724)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:32.468702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:32.516536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4fb7d34a90445e9677ce7287759724', 'request_id': 'req-f8875b74-7904-4d67-8c43-7f7518af6780'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.521914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4fb7d34a90445e9677ce7287759724', 'request_id': 'req-f8875b74-7904-4d67-8c43-7f7518af6780'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.529303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4fb7d34a90445e9677ce7287759724', 'request_id': 'req-f8875b74-7904-4d67-8c43-7f7518af6780'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.540950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] The token cache is being invalidate because project cb4fb7d34a90445e9677ce7287759724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:36:32.541239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.541878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.543357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1162/2301] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:36:32 2022] DELETE /identity/v3/projects/cb4fb7d34a90445e9677ce7287759724 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:36:32.547400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:32.548807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:32.561507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33ecdb-c8e4-49a5-9599-029665193c45 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:32.617305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:32.617933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:32.618211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:32.618482 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] PATH_INFO: `/v3/users/f959af6a61ae46fd8d9e00904194625f` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:32.619618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f959af6a61ae46fd8d9e00904194625f)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:32.621605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:32.644049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:32.670235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f959af6a61ae46fd8d9e00904194625f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.676925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f959af6a61ae46fd8d9e00904194625f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.684297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f959af6a61ae46fd8d9e00904194625f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.684930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f959af6a61ae46fd8d9e00904194625f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.685577 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-57ad45c1-f867-4a14-b460-1b85e7ef9176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f959af6a61ae46fd8d9e00904194625f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:32.702435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1140/2302] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:36:32 2022] DELETE /identity/v3/users/f959af6a61ae46fd8d9e00904194625f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:36:32.879254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:32.880704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:32.897564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8875b74-7904-4d67-8c43-7f7518af6780 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:32.953320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:32.954562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:32.954838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:32.955115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] PATH_INFO: `/v3/projects/a0797133ed164db5bee55fe1578d2aee` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:32.956234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0797133ed164db5bee55fe1578d2aee)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:32.958180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:32.968869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:33.005102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0797133ed164db5bee55fe1578d2aee', 'request_id': 'req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:33.010515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0797133ed164db5bee55fe1578d2aee', 'request_id': 'req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:33.018559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0797133ed164db5bee55fe1578d2aee', 'request_id': 'req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:33.029471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] The token cache is being invalidate because project a0797133ed164db5bee55fe1578d2aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:36:33.029709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:33.030340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4c37f962-f925-4ac9-89e2-2aa3b9c1e96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:36:33.031816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1163/2303] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:36:32 2022] DELETE /identity/v3/projects/a0797133ed164db5bee55fe1578d2aee => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:36:33.036676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a6fb-d6c6-47ef-aa47-a45e74d785e9 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.036962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a6fb-d6c6-47ef-aa47-a45e74d785e9 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.037240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a6fb-d6c6-47ef-aa47-a45e74d785e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.056573 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-89c9a6fb-d6c6-47ef-aa47-a45e74d785e9 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:33.057804 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-89c9a6fb-d6c6-47ef-aa47-a45e74d785e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:33.116309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1141/2304] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:33.124101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312e8ec-6006-4b26-a197-dad788996c14 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.124391 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312e8ec-6006-4b26-a197-dad788996c14 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.124673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312e8ec-6006-4b26-a197-dad788996c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.144188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-0312e8ec-6006-4b26-a197-dad788996c14 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:33.145402 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0312e8ec-6006-4b26-a197-dad788996c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:33.153101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1164/2305] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:33.157609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a6fb-d6c6-47ef-aa47-a45e74d785e9 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.166987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a6fb-d6c6-47ef-aa47-a45e74d785e9 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.171476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.172063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.172961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.173233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.173901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:36:33.174203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.175791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.186953 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1142/2306] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:36:33.191081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312e8ec-6006-4b26-a197-dad788996c14 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.200749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312e8ec-6006-4b26-a197-dad788996c14 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.206126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.206720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.206998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.207269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.207900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.209839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.242704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1165/2307] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:36:33 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:36:33.246888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.256640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d145e-6320-4602-853c-5ef8a35b85bf admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.311739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.312323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.312608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.312886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.313546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.315466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.336442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1143/2308] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:36:33 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:36:33.340503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.351597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874be82-9f93-4171-a5d7-bdd4ef11c528 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.356192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.356781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.357062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.357370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.358017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.360107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.369805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1166/2309] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:36:33.373508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.383851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040674-79a7-4b98-bc98-8e571669c8b6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.388546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff68a1-24d8-4968-aa69-fa401e5d9901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.389136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff68a1-24d8-4968-aa69-fa401e5d9901 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.389460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff68a1-24d8-4968-aa69-fa401e5d9901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.389733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff68a1-24d8-4968-aa69-fa401e5d9901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.390368 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff68a1-24d8-4968-aa69-fa401e5d9901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.392040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff68a1-24d8-4968-aa69-fa401e5d9901 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.402344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1144/2310] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:36:33.406313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.416280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617fea8-7a76-467a-bf99-5921ffa0b228 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.420832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.421508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.421787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.422555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] PATH_INFO: `/v3/projects/f63c30b5d2af4144b096b78c7ab308a7/users/770127af5f494b7e9198438441868fa1/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.437099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63c30b5d2af4144b096b78c7ab308a7, user_id=770127af5f494b7e9198438441868fa1, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.440431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.452999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1167/2311] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:36:33 2022] PUT /identity/v3/projects/f63c30b5d2af4144b096b78c7ab308a7/users/770127af5f494b7e9198438441868fa1/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:36:33.458140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.458426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.458695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.471935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:33.478032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:33.479714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:33.539097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1145/2312] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:33.549072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.550788 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:33.607640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db04fa3-d8b9-4993-a3d4-8d859fa5be0a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.611768 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:33.612023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d784f-b763-48e0-b434-2adffc915a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.612629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d784f-b763-48e0-b434-2adffc915a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.612905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d784f-b763-48e0-b434-2adffc915a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.613162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:33.613288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d784f-b763-48e0-b434-2adffc915a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.622170 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d784f-b763-48e0-b434-2adffc915a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.623591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d784f-b763-48e0-b434-2adffc915a42 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.633284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1168/2313] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:33.688348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8262749-2547-4ab1-b303-7140a06fba17 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:33.694023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:33.694630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:33.695248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:33.695647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:33.697497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:33.764219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:33.766011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:33.775726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1146/2314] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:36:37.744551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da45e2-d357-4de7-a387-6e986eeeacf3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:37.745462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da45e2-d357-4de7-a387-6e986eeeacf3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:37.745954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da45e2-d357-4de7-a387-6e986eeeacf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:37.777722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c9da45e2-d357-4de7-a387-6e986eeeacf3 None None] MFA Rules not processed for user `770127af5f494b7e9198438441868fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:37.778930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c9da45e2-d357-4de7-a387-6e986eeeacf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:37.817856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:37.859517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1169/2315] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:37.864222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:37.874555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eef69-9a7d-46ef-8a33-727049ec76c2 service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:37.879177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:37.879793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:37.880071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:37.880341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:37.881010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:37.883138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:37.911906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1147/2316] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:36:37 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:36:37.915926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da45e2-d357-4de7-a387-6e986eeeacf3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:37.927538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da45e2-d357-4de7-a387-6e986eeeacf3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:37.987613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:37.988221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:37.988496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:37.988765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:37.989521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:37.991492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:38.012286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1170/2317] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:36:37 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:36:38.016906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:38.027624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96724-2ba4-4b50-8ea7-71af635f2f9f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:38.033182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:38.033911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:38.034265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:38.034609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:38.035332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:38.037071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:38.047713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1148/2318] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:36:38.051791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:38.061699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4e43b-4c99-4e55-b9cc-12de03b32fe4 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:38.066420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:38.067048 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:38.067329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:38.067601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] PATH_INFO: `/v3/projects/79e32c4ab8954579906632ff27e2c999/users/602fc05fa9fe4aefb4afa003c16cfeb2/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:38.082521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e32c4ab8954579906632ff27e2c999, user_id=602fc05fa9fe4aefb4afa003c16cfeb2, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:38.085259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:38.099082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1171/2319] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:36:38 2022] PUT /identity/v3/projects/79e32c4ab8954579906632ff27e2c999/users/602fc05fa9fe4aefb4afa003c16cfeb2/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:36:38.103032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:38.113132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e6138-4032-4551-878d-86b6e3b5f229 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:38.122406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:38.158075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:38.159905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:38.175933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:38.176541 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:38.176823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:38.177094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:38.177801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:38.179399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:38.189967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1149/2320] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:36:38.193978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:38.204360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a2d35-cb17-4867-ae05-79c990b6d070 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:38.209449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191cf7a-95fa-43fa-b9d2-fdc494430bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:38.210317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191cf7a-95fa-43fa-b9d2-fdc494430bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:38.210599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191cf7a-95fa-43fa-b9d2-fdc494430bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:38.210875 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191cf7a-95fa-43fa-b9d2-fdc494430bed admin admin] PATH_INFO: `/v3/projects/79e32c4ab8954579906632ff27e2c999/users/602fc05fa9fe4aefb4afa003c16cfeb2/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:38.212812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191cf7a-95fa-43fa-b9d2-fdc494430bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e32c4ab8954579906632ff27e2c999, user_id=602fc05fa9fe4aefb4afa003c16cfeb2, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:38.215491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191cf7a-95fa-43fa-b9d2-fdc494430bed admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:38.224669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41406be-ef7d-4131-ac83-00b3b6357d0e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:38.226593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1150/2321] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:36:38 2022] PUT /identity/v3/projects/79e32c4ab8954579906632ff27e2c999/users/602fc05fa9fe4aefb4afa003c16cfeb2/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:36:38.279074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:38.279680 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:38.279959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:38.280231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:38.282556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:38.287578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:38.355591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:38.357682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:38.378711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1172/2322] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:43.131122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:43.131695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:43.131695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:43.164197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] MFA Rules not processed for user `602fc05fa9fe4aefb4afa003c16cfeb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:43.165458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:43.236486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1151/2323] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:36:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:43.249872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:43.324586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82399a7-52a8-4cbc-881d-96b0fff20bb9 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:43.330113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce486219-cea9-4758-b3e7-c73548b08b6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:43.330725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce486219-cea9-4758-b3e7-c73548b08b6d service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:43.331007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce486219-cea9-4758-b3e7-c73548b08b6d service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:43.331334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce486219-cea9-4758-b3e7-c73548b08b6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:43.389509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce486219-cea9-4758-b3e7-c73548b08b6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:43.391080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce486219-cea9-4758-b3e7-c73548b08b6d service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:43.400597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1173/2324] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:45.395177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:45.398050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:45.420297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618b91e-3bd7-4d19-b37d-b2b01fa3a8f1 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:45.428208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:45.429343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:45.429860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:45.430361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:45.432759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:45.450506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:45.452605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:45.484820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1152/2325] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:53.396207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=81 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:55.460960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:36:59.104689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:59.105188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:59.105313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:59.124757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] MFA Rules not processed for user `e6842565de354b0c85b04940d57c7432`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:36:59.126036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:59.183519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1174/2326] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:36:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:36:59.494583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:59.549096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00213-a357-4abf-81ae-62517f2382a0 service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:59.552707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:59.553328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:59.553615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:59.553882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:59.555623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:59.566270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:59.567712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:59.576113 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1153/2327] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:36:59.886561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:36:59.887951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:59.960776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1347d0-cb81-4a36-b70e-f6e3006e5f54 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:36:59.965704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bbdd4-1647-45b6-8442-1fa4eb8300bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:36:59.966309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbdd4-1647-45b6-8442-1fa4eb8300bb service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:36:59.966587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbdd4-1647-45b6-8442-1fa4eb8300bb service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:36:59.966857 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbdd4-1647-45b6-8442-1fa4eb8300bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:36:59.968566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-db2bbdd4-1647-45b6-8442-1fa4eb8300bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:36:59.980701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bbdd4-1647-45b6-8442-1fa4eb8300bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:36:59.982334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2bbdd4-1647-45b6-8442-1fa4eb8300bb service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:36:59.992722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1175/2328] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:36:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:01.266702 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:01.281146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0a1ec-f0ec-4e52-abf5-f98c0c42112a service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:01.288837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:01.289608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:01.289988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:01.290480 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:01.292313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:01.305197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:01.306892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:01.319270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1154/2329] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:02.266972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:02.267508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:02.267508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:02.287504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] MFA Rules not processed for user `770127af5f494b7e9198438441868fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:02.288796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:02.295935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1176/2330] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:02.401094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:02.413720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35def526-b37c-43c6-8ce9-6d5bab6c21a8 service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:02.418342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:02.419263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:02.419554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:02.419831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:02.422530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:02.434159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:02.435854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:02.455393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1155/2331] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:02.642580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:02.643991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:02.656156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c15c4-ccb6-4b0b-9231-d8ff5cde0b70 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:02.660512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a3e7e-d83d-44ef-b55c-266d3027a245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:02.661188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3e7e-d83d-44ef-b55c-266d3027a245 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:02.661512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3e7e-d83d-44ef-b55c-266d3027a245 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:02.661789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3e7e-d83d-44ef-b55c-266d3027a245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:02.663527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-690a3e7e-d83d-44ef-b55c-266d3027a245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:02.673158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a3e7e-d83d-44ef-b55c-266d3027a245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:02.674669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a3e7e-d83d-44ef-b55c-266d3027a245 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:02.683030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1177/2332] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:02.976627 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:02.993287 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5417f04-65b2-4502-8c1a-e25f7433ef1d service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:03.004798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:03.005618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:03.005862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:03.006250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:03.008403 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:03.122067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:03.123747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:03.136054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1156/2333] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:03.448311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:03.449158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:03.449648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:03.480203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:03.482855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:03.556071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1178/2334] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:37:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:03.569697 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:03.571310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:03.661719 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5c431-5f06-42a7-88ee-c8977ad22f6d service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:03.667419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:03.667981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:03.668278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:03.668527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:03.678092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:03.679787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:03.684801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:03.689147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1157/2335] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:37:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:37:09.146215 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:09.147668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:09.175634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccf5fd-a93b-455f-be98-ea42b48c8f35 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:09.191076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:09.191993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:09.192428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:09.192850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:09.194910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:09.230132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:09.231162 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:09.232847 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:09.246310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:09.261819 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c8404-d584-432b-bcdf-fa2d68149902 service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:09.279015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:09.279735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:09.286625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1179/2336] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:09.301803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:09.301997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] PATH_INFO: `/v3/users/2aa6d6d56f9f4618ae3883c9beb2e77b` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:09.304039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa6d6d56f9f4618ae3883c9beb2e77b)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:09.306012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:09.430017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa6d6d56f9f4618ae3883c9beb2e77b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:09.458489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa6d6d56f9f4618ae3883c9beb2e77b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:09.481583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa6d6d56f9f4618ae3883c9beb2e77b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:09.482676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa6d6d56f9f4618ae3883c9beb2e77b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:09.483604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-03eb426e-1737-4996-aa28-2872d381490f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa6d6d56f9f4618ae3883c9beb2e77b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:09.508534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1158/2337] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:09 2022] DELETE /identity/v3/users/2aa6d6d56f9f4618ae3883c9beb2e77b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:09.974721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:09.990520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075e98a-7d43-4175-a764-98aa8d152e8f service placement] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:09.997407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:10.063106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:10.063831 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.064120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.064395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] PATH_INFO: `/v3/projects/2301f937213e47c29686be4e8601ac32` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.068356 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2301f937213e47c29686be4e8601ac32)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:10.070378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:10.158847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2301f937213e47c29686be4e8601ac32', 'request_id': 'req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:10.166752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2301f937213e47c29686be4e8601ac32', 'request_id': 'req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:10.176618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2301f937213e47c29686be4e8601ac32', 'request_id': 'req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:10.199283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] The token cache is being invalidate because project 2301f937213e47c29686be4e8601ac32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:10.199721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:10.200564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-470fa5a2-e8c5-4c48-9a44-72a87ebb2741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:10.202312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1180/2338] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:09 2022] DELETE /identity/v3/projects/2301f937213e47c29686be4e8601ac32 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:10.210127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72168-ebeb-40eb-9244-7f7df5b2095c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.210568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72168-ebeb-40eb-9244-7f7df5b2095c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.211003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72168-ebeb-40eb-9244-7f7df5b2095c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.242242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-75b72168-ebeb-40eb-9244-7f7df5b2095c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:10.245880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-75b72168-ebeb-40eb-9244-7f7df5b2095c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:10.353184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1159/2339] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:10.361145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a71f88-550f-41db-8d81-b0d387194d2d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.362638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a71f88-550f-41db-8d81-b0d387194d2d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.363251 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a71f88-550f-41db-8d81-b0d387194d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.388722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d4a71f88-550f-41db-8d81-b0d387194d2d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:10.392244 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d4a71f88-550f-41db-8d81-b0d387194d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:10.402617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1181/2340] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:10.406991 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b72168-ebeb-40eb-9244-7f7df5b2095c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:10.424455 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b72168-ebeb-40eb-9244-7f7df5b2095c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:10.428892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:10.430234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.430747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.431229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.432059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:37:10.432370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:10.434051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:10.446737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1160/2341] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:37:10.451542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a71f88-550f-41db-8d81-b0d387194d2d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:10.463020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a71f88-550f-41db-8d81-b0d387194d2d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:10.472047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:10.472562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.473063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.473571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.474413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:10.476668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:10.518289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1182/2342] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:10 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:10.522672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:10.538090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ee35d-6c44-4b1f-b219-9f35e367b38c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:10.612321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:10.612984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.613353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.613638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.614530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:10.617123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:10.626035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:10.650833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1161/2343] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:10 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:10.655103 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:10.668960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becd410-08fa-4b1e-90d2-3c8e3dae8d3b admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:10.673814 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:10.674421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.674692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.675248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.675891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:10.677670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:10.695624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1183/2344] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:10.696823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:10.712213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad69391-d1e1-4a82-9872-72a499821373 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:10.717897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfa3d2-fe11-4190-9d20-17b4b6be4c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:10.718831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfa3d2-fe11-4190-9d20-17b4b6be4c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.719216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfa3d2-fe11-4190-9d20-17b4b6be4c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.720201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfa3d2-fe11-4190-9d20-17b4b6be4c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.721210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadfa3d2-fe11-4190-9d20-17b4b6be4c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:10.723027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadfa3d2-fe11-4190-9d20-17b4b6be4c01 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:10.736453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1162/2345] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:10.740210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:10.757432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979eb4d-9c59-4c34-ad76-fcbe911fffb1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:10.762740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:10.763639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.763922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.764208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] PATH_INFO: `/v3/projects/abadecca422a4c238e7d6c8d25428f05/users/a06a64eddfd64c96a4e002318cd76a8d/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.782794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=abadecca422a4c238e7d6c8d25428f05, user_id=a06a64eddfd64c96a4e002318cd76a8d, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:10.785958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:10.806018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1184/2346] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:10 2022] PUT /identity/v3/projects/abadecca422a4c238e7d6c8d25428f05/users/a06a64eddfd64c96a4e002318cd76a8d/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:10.819658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f0576-2b1a-40a8-bdd6-8a36f25df44e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:10.820424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f0576-2b1a-40a8-bdd6-8a36f25df44e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:10.820704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f0576-2b1a-40a8-bdd6-8a36f25df44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:10.854733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e82f0576-2b1a-40a8-bdd6-8a36f25df44e None None] MFA Rules not processed for user `a06a64eddfd64c96a4e002318cd76a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:10.856204 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e82f0576-2b1a-40a8-bdd6-8a36f25df44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:10.929347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1163/2347] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:10.934038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:10.978470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:11.016775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f61ba-97d9-495e-be58-6f1e448ebabd admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:11.028257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:11.028899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.029186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.029495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.030145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:11.032362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:11.069187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1185/2348] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:10 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:11.070551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f0576-2b1a-40a8-bdd6-8a36f25df44e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:11.085425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f0576-2b1a-40a8-bdd6-8a36f25df44e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:11.216323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:11.217252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.220117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.221972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.222813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:11.228451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:11.312814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1164/2349] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:11 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:11.315519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:11.348977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda98ef2-7313-4e85-8613-7fa4cfbbacbc admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:11.364617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:11.365549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.366001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.366230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.366970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:11.368722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:11.381813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1186/2350] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:11.386530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:11.404399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff6ad9-acb2-4d81-af3f-5cb5075b6c5d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:11.410003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:11.410695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.410974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.411251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] PATH_INFO: `/v3/projects/a2eb3025d8894c53a770e5a92290ebb9/users/d4ba43b5a1a34a4497b31230d8a6d7b3/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.428802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2eb3025d8894c53a770e5a92290ebb9, user_id=d4ba43b5a1a34a4497b31230d8a6d7b3, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:11.432078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:11.456157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1165/2351] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:11 2022] PUT /identity/v3/projects/a2eb3025d8894c53a770e5a92290ebb9/users/d4ba43b5a1a34a4497b31230d8a6d7b3/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:11.460090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:11.473511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed40a7-7605-41d4-81a6-f2bb6d3ee0ef admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:11.559989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb87f0-34a7-49f4-af78-118914697d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:11.560611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb87f0-34a7-49f4-af78-118914697d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.560887 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb87f0-34a7-49f4-af78-118914697d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.561315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb87f0-34a7-49f4-af78-118914697d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.562187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb87f0-34a7-49f4-af78-118914697d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:11.563965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb87f0-34a7-49f4-af78-118914697d3b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:11.580845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1187/2352] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:11.585306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:11.597132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7d629-e542-445f-a2e3-331648f2404b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:11.602139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:11.602851 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.603128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.603400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] PATH_INFO: `/v3/projects/a2eb3025d8894c53a770e5a92290ebb9/users/d4ba43b5a1a34a4497b31230d8a6d7b3/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.605426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2eb3025d8894c53a770e5a92290ebb9, user_id=d4ba43b5a1a34a4497b31230d8a6d7b3, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:11.608116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:11.620912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1166/2353] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:11 2022] PUT /identity/v3/projects/a2eb3025d8894c53a770e5a92290ebb9/users/d4ba43b5a1a34a4497b31230d8a6d7b3/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:11.632324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.632324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.632324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.664121 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] MFA Rules not processed for user `d4ba43b5a1a34a4497b31230d8a6d7b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:11.665400 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:11.745203 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1188/2354] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:11.757140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:11.761506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:11.790630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:11.830264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b264c-f8ff-41a2-baf8-eb0bcbcad377 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:11.833908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:11.834475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:11.834814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:11.834992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:11.859807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:11.861630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:11.873671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1167/2355] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:12.025148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:12.026931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:12.063302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246abd2-d35e-45a9-9d91-949f4292b28e None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:12.071606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:12.072234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:12.072510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:12.075009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:12.155542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:12.205829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:12.207423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:12.221939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1189/2356] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:14.319978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:14.322902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:14.423164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce40ba-3f03-41fe-859d-c19d93075b6a service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:14.444085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a67e43-ebb8-4686-b63a-6c55b7b36d71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:14.444943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a67e43-ebb8-4686-b63a-6c55b7b36d71 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:14.445237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a67e43-ebb8-4686-b63a-6c55b7b36d71 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:14.445618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a67e43-ebb8-4686-b63a-6c55b7b36d71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:14.449365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d6a67e43-ebb8-4686-b63a-6c55b7b36d71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:14.465828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a67e43-ebb8-4686-b63a-6c55b7b36d71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:14.467800 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a67e43-ebb8-4686-b63a-6c55b7b36d71 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:14.484431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1168/2357] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:22.216318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=15 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:22.718029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:22.779582 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed79fe7-01ed-475b-afd5-e79289a35ac5 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:22.784513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:22.785135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:22.785452 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:22.785727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] PATH_INFO: `/v3/users/a06a64eddfd64c96a4e002318cd76a8d` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:22.786852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06a64eddfd64c96a4e002318cd76a8d)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:22.788776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:22.833242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06a64eddfd64c96a4e002318cd76a8d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:22.840584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06a64eddfd64c96a4e002318cd76a8d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:22.841492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06a64eddfd64c96a4e002318cd76a8d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:22.842134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06a64eddfd64c96a4e002318cd76a8d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:22.850885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06a64eddfd64c96a4e002318cd76a8d'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:22.869621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1190/2358] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:22 2022] DELETE /identity/v3/users/a06a64eddfd64c96a4e002318cd76a8d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:22.874310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261587-6840-4562-9f89-111cd79a2432 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:22.874877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261587-6840-4562-9f89-111cd79a2432 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:22.874988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261587-6840-4562-9f89-111cd79a2432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:22.894709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-38261587-6840-4562-9f89-111cd79a2432 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:22.895917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-38261587-6840-4562-9f89-111cd79a2432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:22.956325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1169/2359] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:37:22.965999 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:22.967639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:23.032322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d4daf-404e-47ee-8b4a-951fed276180 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:23.035891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:23.036519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:23.036823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:23.037210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:23.047187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:23.048628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:23.058243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1191/2360] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:23.326818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38261587-6840-4562-9f89-111cd79a2432 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:23.328317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-38261587-6840-4562-9f89-111cd79a2432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:23.341619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38261587-6840-4562-9f89-111cd79a2432 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:23.347292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:23.347961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:23.348245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:23.348519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] PATH_INFO: `/v3/projects/abadecca422a4c238e7d6c8d25428f05` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:23.349694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abadecca422a4c238e7d6c8d25428f05)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:23.351631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:23.417945 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:23.422947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abadecca422a4c238e7d6c8d25428f05', 'request_id': 'req-853d940c-5841-4271-9412-1d4da9d95411'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.429068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abadecca422a4c238e7d6c8d25428f05', 'request_id': 'req-853d940c-5841-4271-9412-1d4da9d95411'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.436313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abadecca422a4c238e7d6c8d25428f05', 'request_id': 'req-853d940c-5841-4271-9412-1d4da9d95411'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.447099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] The token cache is being invalidate because project abadecca422a4c238e7d6c8d25428f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:23.447382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.448011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.449506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1170/2361] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:23 2022] DELETE /identity/v3/projects/abadecca422a4c238e7d6c8d25428f05 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:23.454040 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:23.463442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b344e-b411-41b3-8cec-d521cca457d2 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:23.482485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:23.544888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:23.545655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:23.545935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:23.546235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] PATH_INFO: `/v3/users/d4ba43b5a1a34a4497b31230d8a6d7b3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:23.547526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ba43b5a1a34a4497b31230d8a6d7b3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:23.549554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:23.604191 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ba43b5a1a34a4497b31230d8a6d7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.614730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ba43b5a1a34a4497b31230d8a6d7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.615351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ba43b5a1a34a4497b31230d8a6d7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.615946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ba43b5a1a34a4497b31230d8a6d7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.623073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-cc590956-7062-4fd2-9399-a587397a2711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ba43b5a1a34a4497b31230d8a6d7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:23.639951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1192/2362] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:23 2022] DELETE /identity/v3/users/d4ba43b5a1a34a4497b31230d8a6d7b3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:23.930893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:23.933072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:23.948088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d940c-5841-4271-9412-1d4da9d95411 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.004546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.005175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.005514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.005792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] PATH_INFO: `/v3/projects/a2eb3025d8894c53a770e5a92290ebb9` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.006891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2eb3025d8894c53a770e5a92290ebb9)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.008825 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.055159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2eb3025d8894c53a770e5a92290ebb9', 'request_id': 'req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:24.060870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2eb3025d8894c53a770e5a92290ebb9', 'request_id': 'req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:24.069274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2eb3025d8894c53a770e5a92290ebb9', 'request_id': 'req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:24.081283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] The token cache is being invalidate because project a2eb3025d8894c53a770e5a92290ebb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:24.081585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:24.082237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-28cd5d63-a74a-4eb0-9d90-46d23e69e96f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:24.083741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1171/2363] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:23 2022] DELETE /identity/v3/projects/a2eb3025d8894c53a770e5a92290ebb9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:24.088610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510e74-735f-4bd3-b9dd-f403094a7659 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.089148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510e74-735f-4bd3-b9dd-f403094a7659 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.089459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510e74-735f-4bd3-b9dd-f403094a7659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.110485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-12510e74-735f-4bd3-b9dd-f403094a7659 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:24.111686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-12510e74-735f-4bd3-b9dd-f403094a7659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:24.183506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1193/2364] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:24.192677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f8f0-a852-49cf-8d08-b8fba29a7e8e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.193086 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f8f0-a852-49cf-8d08-b8fba29a7e8e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.193338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f8f0-a852-49cf-8d08-b8fba29a7e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.218221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4fb7f8f0-a852-49cf-8d08-b8fba29a7e8e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:24.219467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4fb7f8f0-a852-49cf-8d08-b8fba29a7e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:24.227801 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1172/2365] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:24.232147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510e74-735f-4bd3-b9dd-f403094a7659 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.242303 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510e74-735f-4bd3-b9dd-f403094a7659 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.246768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.247359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.247634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.247902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.248538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:37:24.248853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.251004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.265321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1194/2366] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:37:24.270029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7f8f0-a852-49cf-8d08-b8fba29a7e8e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.280661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7f8f0-a852-49cf-8d08-b8fba29a7e8e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.288087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.288709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.288987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.289260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.289958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.292156 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.325017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1173/2367] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:24 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:24.329118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.340431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d939c7-8985-4064-b4eb-f3412a8c6c2a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.369959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:24.406728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.407224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.407507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.407779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.408467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.410658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.435571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1195/2368] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:24 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:24.440124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.454619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced6e4a-90a8-4ac8-8c18-da19035cc53e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.459735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.460351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.460632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.460899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.462263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.463868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.475187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1174/2369] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:37:24.479312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.490959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fb0c0-9c3f-4477-ba0b-c4a287d9eb4a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.495969 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135492c-607c-4067-99b5-a676690cba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.496636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135492c-607c-4067-99b5-a676690cba3c admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.497010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135492c-607c-4067-99b5-a676690cba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.497367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135492c-607c-4067-99b5-a676690cba3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.498157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135492c-607c-4067-99b5-a676690cba3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.500698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135492c-607c-4067-99b5-a676690cba3c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.518458 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1196/2370] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:24.522579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.539796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec1316-8d85-4b38-ab34-204ca0166493 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.546264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.547551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.547995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.548318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] PATH_INFO: `/v3/projects/ae7db86ff8c946bb94f3253420c29465/users/535f6149254542daba6f260ffa227843/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.571708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7db86ff8c946bb94f3253420c29465, user_id=535f6149254542daba6f260ffa227843, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.580999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.590065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:24.600812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1175/2371] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:24 2022] PUT /identity/v3/projects/ae7db86ff8c946bb94f3253420c29465/users/535f6149254542daba6f260ffa227843/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:24.612060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0202-e0f4-4b72-bc9a-5ad1bba9d71e None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.612357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0202-e0f4-4b72-bc9a-5ad1bba9d71e None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.612632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0202-e0f4-4b72-bc9a-5ad1bba9d71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.643350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6cac0202-e0f4-4b72-bc9a-5ad1bba9d71e None None] MFA Rules not processed for user `535f6149254542daba6f260ffa227843`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:24.644754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6cac0202-e0f4-4b72-bc9a-5ad1bba9d71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:24.724053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1197/2372] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:24.728161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.804795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0287-a441-4c9c-8326-51d931a2f81e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.811335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.812740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.813211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.813731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.814660 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.817242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:24.861710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1176/2373] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:24 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:37:24.863286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac0202-e0f4-4b72-bc9a-5ad1bba9d71e None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:24.884139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac0202-e0f4-4b72-bc9a-5ad1bba9d71e None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:24.985067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:24.986216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:24.986654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:24.987082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:24.987881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:24.991315 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.021168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1198/2374] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:24 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:25.023060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:25.035209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae9165-0e44-4c4a-80e1-7299a7ac36c0 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:25.041527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:25.042865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.043283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.043635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.044361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:25.046215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.058574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1177/2375] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:25.062897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:25.073444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb63b-67cf-4920-9790-23da3c1eb834 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:25.078752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:25.079406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.079685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.080014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] PATH_INFO: `/v3/projects/87956c96257e45ec8d7f7796b0cdfad3/users/d8e027d594e94b9e85e4cfa2590e5235/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.096642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87956c96257e45ec8d7f7796b0cdfad3, user_id=d8e027d594e94b9e85e4cfa2590e5235, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:25.099349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.116869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1199/2376] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:25 2022] PUT /identity/v3/projects/87956c96257e45ec8d7f7796b0cdfad3/users/d8e027d594e94b9e85e4cfa2590e5235/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:25.118575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:25.128725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c6f74-349f-4730-b7a2-1dfdcfe27007 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:25.198626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dc175-c0b1-4734-aced-1bff49318dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:25.199383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc175-c0b1-4734-aced-1bff49318dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.199773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc175-c0b1-4734-aced-1bff49318dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.200167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dc175-c0b1-4734-aced-1bff49318dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.201074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dc175-c0b1-4734-aced-1bff49318dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:25.206612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dc175-c0b1-4734-aced-1bff49318dd4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.221565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1178/2377] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:37:25.224291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:25.229942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:25.236182 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42cd1f-bcf5-4ac6-818f-d38ac214ad8e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:25.242426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:25.242816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.243787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.243957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] PATH_INFO: `/v3/projects/87956c96257e45ec8d7f7796b0cdfad3/users/d8e027d594e94b9e85e4cfa2590e5235/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.245931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87956c96257e45ec8d7f7796b0cdfad3, user_id=d8e027d594e94b9e85e4cfa2590e5235, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:25.248682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.266375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1200/2378] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:25 2022] PUT /identity/v3/projects/87956c96257e45ec8d7f7796b0cdfad3/users/d8e027d594e94b9e85e4cfa2590e5235/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:25.277230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.278233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.278651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.294590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:25.296323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:25.314758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] MFA Rules not processed for user `d8e027d594e94b9e85e4cfa2590e5235`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:25.316655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:25.366762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a4c11-d102-4df9-9927-69c6add1f10d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:25.371088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:25.371893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.372309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.372725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.375329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:25.382925 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:25.404314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1179/2379] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:25.411841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:25.413568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:25.478293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab8b1-1b37-4440-8583-4ca6f20ce5ae None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:25.483434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:25.484095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.484386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.484680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.486352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:25.487987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.498745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:25.500987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1201/2380] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:37:25.501372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.511706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1180/2381] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:25.563059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:25.564550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:25.579819 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff69950-0f90-4e08-a435-961f767e22be service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:25.583600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343dac2-e7cb-4632-9747-ba33b55a0920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:25.584217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343dac2-e7cb-4632-9747-ba33b55a0920 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:25.584496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343dac2-e7cb-4632-9747-ba33b55a0920 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:25.584768 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343dac2-e7cb-4632-9747-ba33b55a0920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:25.659888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343dac2-e7cb-4632-9747-ba33b55a0920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:25.661934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343dac2-e7cb-4632-9747-ba33b55a0920 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:25.673446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1202/2382] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:29.369324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:29.495096 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fa43f-4be0-4b15-938a-8a113c5416f9 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:29.501047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:29.501837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:29.502241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:29.502636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] PATH_INFO: `/v3/users/535f6149254542daba6f260ffa227843` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:29.504466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=535f6149254542daba6f260ffa227843)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:29.506108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:29.521863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be93f8-4f65-4f16-b302-9222928954dc None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:29.522447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be93f8-4f65-4f16-b302-9222928954dc None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:29.522447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be93f8-4f65-4f16-b302-9222928954dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:29.543481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b6be93f8-4f65-4f16-b302-9222928954dc None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:29.544806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b6be93f8-4f65-4f16-b302-9222928954dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:29.569502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535f6149254542daba6f260ffa227843'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:29.580325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535f6149254542daba6f260ffa227843'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:29.584794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:29.589292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535f6149254542daba6f260ffa227843'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:29.590159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535f6149254542daba6f260ffa227843'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:29.591840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535f6149254542daba6f260ffa227843'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:29.614024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1203/2383] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:29.621742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1181/2384] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:29 2022] DELETE /identity/v3/users/535f6149254542daba6f260ffa227843 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:29.631822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:29.632188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:29.632350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:29.638657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:29.640677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:29.657725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:29.659317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:29.663465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:29.738157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1204/2385] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:29.745651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f3f6b-6942-49fa-9828-3e1f81f04f4b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:29.750536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:29.751536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:29.752436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:29.752717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:29.752986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:29.754016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:29.825510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:29.827159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:29.841542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1182/2386] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:37:29.846387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f7cab-3e65-4074-97a2-2bd3ff454c18 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:29.852374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:29.853109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:29.853544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:29.853920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:29.866888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:29.868912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:29.883515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1205/2387] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:37:30.444644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:30.446683 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:30.472609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f46c4-9ab3-4d60-bffe-62c365e3063d service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:30.483244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:30.484385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:30.484676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:30.484952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] PATH_INFO: `/v3/projects/ae7db86ff8c946bb94f3253420c29465` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:30.490998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7db86ff8c946bb94f3253420c29465)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:30.493112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:30.578377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7db86ff8c946bb94f3253420c29465', 'request_id': 'req-374046a2-4fc7-445b-ae2a-4ace2713a9f4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.600094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7db86ff8c946bb94f3253420c29465', 'request_id': 'req-374046a2-4fc7-445b-ae2a-4ace2713a9f4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.613410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7db86ff8c946bb94f3253420c29465', 'request_id': 'req-374046a2-4fc7-445b-ae2a-4ace2713a9f4'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.627092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] The token cache is being invalidate because project ae7db86ff8c946bb94f3253420c29465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:30.627383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.628051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.629607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1183/2388] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:30 2022] DELETE /identity/v3/projects/ae7db86ff8c946bb94f3253420c29465 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:30.633715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:30.644335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b42b1-5357-46bc-8840-b54fc1c5741e service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:30.777940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:30.778603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:30.778883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:30.779157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] PATH_INFO: `/v3/users/d8e027d594e94b9e85e4cfa2590e5235` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:30.783864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e027d594e94b9e85e4cfa2590e5235)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:30.788482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:30.855732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:30.889853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e027d594e94b9e85e4cfa2590e5235'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.908589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e027d594e94b9e85e4cfa2590e5235'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.909357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e027d594e94b9e85e4cfa2590e5235'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.910163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e027d594e94b9e85e4cfa2590e5235'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.919309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-56d10124-198b-450d-9ba0-06868b455861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e027d594e94b9e85e4cfa2590e5235'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:30.946236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1206/2389] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:30 2022] DELETE /identity/v3/users/d8e027d594e94b9e85e4cfa2590e5235 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:31.289722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:31.291303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:31.315620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374046a2-4fc7-445b-ae2a-4ace2713a9f4 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:31.371665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:31.401761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:31.402998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.403394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.403826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] PATH_INFO: `/v3/projects/87956c96257e45ec8d7f7796b0cdfad3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.405637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87956c96257e45ec8d7f7796b0cdfad3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:31.407656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:31.462014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87956c96257e45ec8d7f7796b0cdfad3', 'request_id': 'req-c5b0bf26-4226-4637-9374-9b0282622f80'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:31.470018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87956c96257e45ec8d7f7796b0cdfad3', 'request_id': 'req-c5b0bf26-4226-4637-9374-9b0282622f80'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:31.480654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87956c96257e45ec8d7f7796b0cdfad3', 'request_id': 'req-c5b0bf26-4226-4637-9374-9b0282622f80'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:31.493584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] The token cache is being invalidate because project 87956c96257e45ec8d7f7796b0cdfad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:31.493877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:31.494663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c5b0bf26-4226-4637-9374-9b0282622f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:31.496555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1184/2390] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:31 2022] DELETE /identity/v3/projects/87956c96257e45ec8d7f7796b0cdfad3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:31.502699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95ef9c-4e69-440c-a944-ea62d7c51cfd None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.502990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95ef9c-4e69-440c-a944-ea62d7c51cfd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.503264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95ef9c-4e69-440c-a944-ea62d7c51cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.523591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ad95ef9c-4e69-440c-a944-ea62d7c51cfd None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:31.524784 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ad95ef9c-4e69-440c-a944-ea62d7c51cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:31.596538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1207/2391] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:37:31.605420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dd383-89d8-4274-8eba-6bdf25381294 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.605715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dd383-89d8-4274-8eba-6bdf25381294 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.606019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dd383-89d8-4274-8eba-6bdf25381294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.631597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e15dd383-89d8-4274-8eba-6bdf25381294 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:31.632952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e15dd383-89d8-4274-8eba-6bdf25381294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:31.643263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1185/2392] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:31.653536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95ef9c-4e69-440c-a944-ea62d7c51cfd None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:31.664204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95ef9c-4e69-440c-a944-ea62d7c51cfd None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:31.668924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:31.669557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.669839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.670116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.670775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:37:31.671075 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:31.672733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:31.684530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1208/2393] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:37:31.689597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dd383-89d8-4274-8eba-6bdf25381294 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:31.699144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dd383-89d8-4274-8eba-6bdf25381294 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:31.704554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:31.705172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.705483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.705761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.706395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:31.708326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:31.742242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1186/2394] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:31 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:37:31.746409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:31.757753 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5ddce-d1b7-493f-a750-92cf26506ca8 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:31.820284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:31.820902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.821185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.821494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.822135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:31.824119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:31.857761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1209/2395] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:31 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:31.859478 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:31.871471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99a15f-3522-43fb-8bc0-bba053e52b52 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:31.879371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:31.879988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.880264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.880552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.881852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:31.883493 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:31.897252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1187/2396] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:31.898646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:31.912884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8645d39-c156-44be-b44c-9d880e92a6bc admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:31.917933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7e681-1584-4f0f-ba3c-b0bda9d92567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:31.918628 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e681-1584-4f0f-ba3c-b0bda9d92567 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.918913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e681-1584-4f0f-ba3c-b0bda9d92567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.919187 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e681-1584-4f0f-ba3c-b0bda9d92567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.919836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e681-1584-4f0f-ba3c-b0bda9d92567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:31.922097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e681-1584-4f0f-ba3c-b0bda9d92567 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:31.937933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1210/2397] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:31.942219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:31.952285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f084e-1d2c-4cfc-8b0d-725858170933 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:31.957373 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:31.958045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:31.958334 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:31.958607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] PATH_INFO: `/v3/projects/dc63273658c04c889c754808fbb773de/users/3f5d4db5ac884ecf8d8f906e7fa67288/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:31.977460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc63273658c04c889c754808fbb773de, user_id=3f5d4db5ac884ecf8d8f906e7fa67288, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.000437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.021653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1188/2398] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:31 2022] PUT /identity/v3/projects/dc63273658c04c889c754808fbb773de/users/3f5d4db5ac884ecf8d8f906e7fa67288/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:32.035442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b8963-e4b1-4153-a954-8723f2a0cd6b None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.036039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b8963-e4b1-4153-a954-8723f2a0cd6b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.036417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b8963-e4b1-4153-a954-8723f2a0cd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.065439 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:32.076869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e82b8963-e4b1-4153-a954-8723f2a0cd6b None None] MFA Rules not processed for user `3f5d4db5ac884ecf8d8f906e7fa67288`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:32.079937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e82b8963-e4b1-4153-a954-8723f2a0cd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:32.160825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1211/2399] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:32.167805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:32.267365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30911414-51e8-4e9d-97c8-ab6424406b80 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:32.278238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:32.278862 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.279165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.279408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.280049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.282796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.296195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:32.322150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1189/2400] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:32 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:32.326204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b8963-e4b1-4153-a954-8723f2a0cd6b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:32.340231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b8963-e4b1-4153-a954-8723f2a0cd6b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:32.409949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:32.410589 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.410868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.411228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.411908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.414002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.443168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1212/2401] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:32 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:32.447342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:32.458935 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935ebc1-655c-4132-a3f8-25230fb75761 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:32.463746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:32.464353 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.464615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.464885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.465552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.467184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.478709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1190/2402] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:32.483059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:32.494065 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068fcb3-e767-4d81-b3a3-afb221fac2da admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:32.498938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:32.499648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.499931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.500206 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] PATH_INFO: `/v3/projects/ae375bae421a498dba3411a6872b3565/users/d9819e5b0d5a4694956a96c427f7db9f/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.517883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae375bae421a498dba3411a6872b3565, user_id=d9819e5b0d5a4694956a96c427f7db9f, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.520709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.536209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1213/2403] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:32 2022] PUT /identity/v3/projects/ae375bae421a498dba3411a6872b3565/users/d9819e5b0d5a4694956a96c427f7db9f/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:32.540291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:32.551038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b2d6-3a07-467c-8a30-326ece7b5ea6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:32.632294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e896874-b274-47d5-b8e1-2da6a11c3f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:32.633379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e896874-b274-47d5-b8e1-2da6a11c3f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.633888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e896874-b274-47d5-b8e1-2da6a11c3f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.634225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e896874-b274-47d5-b8e1-2da6a11c3f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.635288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e896874-b274-47d5-b8e1-2da6a11c3f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.637784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e896874-b274-47d5-b8e1-2da6a11c3f7f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.651030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1191/2404] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:32.655368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:32.669048 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2d970-1a5a-460c-a885-c7ba8fbdb884 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:32.674738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:32.675419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.675694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.675964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] PATH_INFO: `/v3/projects/ae375bae421a498dba3411a6872b3565/users/d9819e5b0d5a4694956a96c427f7db9f/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.677884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae375bae421a498dba3411a6872b3565, user_id=d9819e5b0d5a4694956a96c427f7db9f, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.680549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.691496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1214/2405] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:32 2022] PUT /identity/v3/projects/ae375bae421a498dba3411a6872b3565/users/d9819e5b0d5a4694956a96c427f7db9f/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:32.705533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d203a8e-43b8-43e7-8a96-f890f2b4e832 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.705931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d203a8e-43b8-43e7-8a96-f890f2b4e832 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.706306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d203a8e-43b8-43e7-8a96-f890f2b4e832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.739186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5d203a8e-43b8-43e7-8a96-f890f2b4e832 None None] MFA Rules not processed for user `d9819e5b0d5a4694956a96c427f7db9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:32.740930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5d203a8e-43b8-43e7-8a96-f890f2b4e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:32.817632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1192/2406] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:32.828487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:32.830323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:32.889913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10aa8-fde4-474b-83dc-31ea782e4dcb admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:32.894542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:32.895254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:32.895592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:32.895939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:32.904371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:32.907332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:32.909221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:32.920944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1215/2407] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:33.006532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d203a8e-43b8-43e7-8a96-f890f2b4e832 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:33.019166 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d203a8e-43b8-43e7-8a96-f890f2b4e832 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:33.023549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:33.024221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:33.024512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:33.024782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:33.092728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:33.094974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:33.101828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:33.114400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1193/2408] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:34.894617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:34.896861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:34.961403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b9b3-e672-455c-8c6f-0a54d050c4ca service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:34.965116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8859761-e539-4ee7-b4a7-47e5f0203a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:34.965773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8859761-e539-4ee7-b4a7-47e5f0203a1f service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:34.966077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8859761-e539-4ee7-b4a7-47e5f0203a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:34.966375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8859761-e539-4ee7-b4a7-47e5f0203a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:34.968452 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f8859761-e539-4ee7-b4a7-47e5f0203a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:35.038940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8859761-e539-4ee7-b4a7-47e5f0203a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:35.040389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8859761-e539-4ee7-b4a7-47e5f0203a1f service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:35.051938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1216/2409] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:38.788215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:38.799780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5876a4-63a0-4cb0-9e14-692900b7c281 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:38.804161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:38.804772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:38.805199 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:38.805537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] PATH_INFO: `/v3/users/3f5d4db5ac884ecf8d8f906e7fa67288` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:38.806643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f5d4db5ac884ecf8d8f906e7fa67288)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:38.808561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:38.860269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f5d4db5ac884ecf8d8f906e7fa67288'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:38.867782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f5d4db5ac884ecf8d8f906e7fa67288'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:38.875749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f5d4db5ac884ecf8d8f906e7fa67288'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:38.876834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f5d4db5ac884ecf8d8f906e7fa67288'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:38.877617 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f5d4db5ac884ecf8d8f906e7fa67288'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:38.899333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1194/2410] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:38 2022] DELETE /identity/v3/users/3f5d4db5ac884ecf8d8f906e7fa67288 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:38.903665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:38.904246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:38.904520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:38.926200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:38.927699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.025924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1217/2411] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:39.043244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:39.046866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.118078 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c4f9a-e32e-4d8b-8d42-539e53f94d3a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:39.122372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:39.123068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.123409 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.123703 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.133209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:39.135147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:39.144814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1195/2412] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:37:39.288904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:39.290502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.304330 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b583fe-8666-4666-9ca5-3bcd53115463 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:39.309311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:39.309951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.310287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.311173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] PATH_INFO: `/v3/users/9b8e4aad09814f8299a334639e34d948` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.312564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8e4aad09814f8299a334639e34d948)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:39.314657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:39.363516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:39.369728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8e4aad09814f8299a334639e34d948'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.377609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8e4aad09814f8299a334639e34d948'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.378334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8e4aad09814f8299a334639e34d948'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.379358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8e4aad09814f8299a334639e34d948'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.387119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8e4aad09814f8299a334639e34d948'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.407801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1218/2413] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:39 2022] DELETE /identity/v3/users/9b8e4aad09814f8299a334639e34d948 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:39.462237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:39.463913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.482528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cfa1c-1edf-45ce-b8bc-9ddc69496c4e service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:39.547224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:39.547856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.548136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.548411 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] PATH_INFO: `/v3/projects/dc63273658c04c889c754808fbb773de` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.550093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc63273658c04c889c754808fbb773de)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:39.552048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:39.571407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:39.596791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:39.598343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.611864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce084607-5860-4fc7-b5cd-d1773b3e4995 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:39.611984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc63273658c04c889c754808fbb773de', 'request_id': 'req-06b9dc16-480c-41f1-87d4-fbecf108c963'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.619466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc63273658c04c889c754808fbb773de', 'request_id': 'req-06b9dc16-480c-41f1-87d4-fbecf108c963'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.630581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc63273658c04c889c754808fbb773de', 'request_id': 'req-06b9dc16-480c-41f1-87d4-fbecf108c963'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.647475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] The token cache is being invalidate because project dc63273658c04c889c754808fbb773de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:39.647849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.648448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.650444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1196/2414] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:39 2022] DELETE /identity/v3/projects/dc63273658c04c889c754808fbb773de => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:39.654794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:39.656236 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.681681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:39.682311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.682592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.682865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] PATH_INFO: `/v3/projects/0bbb137a895941b1876eebfc56c28425` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.684006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bbb137a895941b1876eebfc56c28425)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:39.685953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:39.720079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9dc16-480c-41f1-87d4-fbecf108c963 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:39.732197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bbb137a895941b1876eebfc56c28425', 'request_id': 'req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.737908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bbb137a895941b1876eebfc56c28425', 'request_id': 'req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.745631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bbb137a895941b1876eebfc56c28425', 'request_id': 'req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.757681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] The token cache is being invalidate because project 0bbb137a895941b1876eebfc56c28425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:39.757965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.758635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-33b97147-d2d0-45a3-b5ab-d6b8b11af9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.760600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1219/2415] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:39 2022] DELETE /identity/v3/projects/0bbb137a895941b1876eebfc56c28425 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:39.766281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dccd4c-36a1-4bf3-ab57-3f09f2e579eb None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.766569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dccd4c-36a1-4bf3-ab57-3f09f2e579eb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.766844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dccd4c-36a1-4bf3-ab57-3f09f2e579eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.777386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:39.778025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.778300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.778571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] PATH_INFO: `/v3/users/d9819e5b0d5a4694956a96c427f7db9f` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.779743 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9819e5b0d5a4694956a96c427f7db9f)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:39.781731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:39.786196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-17dccd4c-36a1-4bf3-ab57-3f09f2e579eb None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:39.787401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-17dccd4c-36a1-4bf3-ab57-3f09f2e579eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.795165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1220/2416] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:37:39.803526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cca26-5add-4876-b875-b034180b8582 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.803810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cca26-5add-4876-b875-b034180b8582 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.804082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cca26-5add-4876-b875-b034180b8582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.823990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d58cca26-5add-4876-b875-b034180b8582 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:39.825147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d58cca26-5add-4876-b875-b034180b8582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.834026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1221/2417] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:37:39.834905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9819e5b0d5a4694956a96c427f7db9f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.838275 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cca26-5add-4876-b875-b034180b8582 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:39.840501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:39.842633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9819e5b0d5a4694956a96c427f7db9f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.848646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cca26-5add-4876-b875-b034180b8582 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:39.851635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9819e5b0d5a4694956a96c427f7db9f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.852513 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9819e5b0d5a4694956a96c427f7db9f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.853549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9819e5b0d5a4694956a96c427f7db9f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:39.853991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:39.854588 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.854872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.855174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.855826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:37:39.856127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:39.857803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:39.869537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1222/2418] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:37:39.871522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1197/2419] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:37:39 2022] DELETE /identity/v3/users/d9819e5b0d5a4694956a96c427f7db9f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:39.876011 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:39.877662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:39.890747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd7910-02d2-4d89-aff1-400da2ff20c0 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:39.909583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:39.954834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:39.955489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:39.955825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:39.956162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:39.956900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:39.958964 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:39.987960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1223/2420] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:39 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:39.992406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.004127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1171e58-734c-4304-b4b0-48f980c12f4b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.068084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.068729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.069011 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.069355 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.070022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.071960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.097126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1198/2421] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:39 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:40.098519 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.109364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72ae92-8ca7-41bc-b771-ebd14a84d628 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.113975 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.114639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.114982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.115316 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.116066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.117861 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.128328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1224/2422] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:40.134385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.146173 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3e26a-e429-4cbe-a66a-962c9299a756 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.150842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.151447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.151724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.151994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.152636 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.154272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.164410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1199/2423] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:40.168557 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.179150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bc69d-483c-4a1f-a3fe-c7f86fe1ec1b admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.183700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a802500-6283-4183-ae3f-10975b207fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.184349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a802500-6283-4183-ae3f-10975b207fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.184627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a802500-6283-4183-ae3f-10975b207fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.184899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a802500-6283-4183-ae3f-10975b207fe1 admin admin] PATH_INFO: `/v3/projects/e9319a14d6ae414d9fb614141dc410b9/users/c7b456384631453989464d7114b0d4e2/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.200495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a802500-6283-4183-ae3f-10975b207fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9319a14d6ae414d9fb614141dc410b9, user_id=c7b456384631453989464d7114b0d4e2, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.203299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a802500-6283-4183-ae3f-10975b207fe1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.219633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.222005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.225177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1225/2424] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:40 2022] PUT /identity/v3/projects/e9319a14d6ae414d9fb614141dc410b9/users/c7b456384631453989464d7114b0d4e2/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:37:40.226638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005b2b0-cdde-4557-86a0-ff12510789bd None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.226923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005b2b0-cdde-4557-86a0-ff12510789bd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.227220 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005b2b0-cdde-4557-86a0-ff12510789bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.242873 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656f6db-1047-4f69-9f95-196a0785c537 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.257931 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d005b2b0-cdde-4557-86a0-ff12510789bd None None] MFA Rules not processed for user `c7b456384631453989464d7114b0d4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:40.259143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d005b2b0-cdde-4557-86a0-ff12510789bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.304245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.304961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.305247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.305553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] PATH_INFO: `/v3/projects/ae375bae421a498dba3411a6872b3565` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.306690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae375bae421a498dba3411a6872b3565)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.308604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.325895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1226/2425] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:40.330252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005b2b0-cdde-4557-86a0-ff12510789bd None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.340981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005b2b0-cdde-4557-86a0-ff12510789bd None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.347146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.347774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.348053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.348321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.348944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.350903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.359343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae375bae421a498dba3411a6872b3565', 'request_id': 'req-ff6e1591-7302-446f-b283-8315cc86f41f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:40.361306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:40.366170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae375bae421a498dba3411a6872b3565', 'request_id': 'req-ff6e1591-7302-446f-b283-8315cc86f41f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:40.375828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae375bae421a498dba3411a6872b3565', 'request_id': 'req-ff6e1591-7302-446f-b283-8315cc86f41f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:40.379944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1227/2426] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:37:40.384046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.389176 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] The token cache is being invalidate because project ae375bae421a498dba3411a6872b3565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:37:40.389990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:40.390676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ff6e1591-7302-446f-b283-8315cc86f41f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:37:40.397249 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1200/2427] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:37:40 2022] DELETE /identity/v3/projects/ae375bae421a498dba3411a6872b3565 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:40.398892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fe0f9-a5d7-437b-b43e-4587ba2681ec None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.399447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fe0f9-a5d7-437b-b43e-4587ba2681ec None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.399750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fe0f9-a5d7-437b-b43e-4587ba2681ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.428715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5a0fe0f9-a5d7-437b-b43e-4587ba2681ec None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:40.429965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5a0fe0f9-a5d7-437b-b43e-4587ba2681ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.435869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:40.458009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f473197-799a-40c6-a5d9-e08a6205ad99 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.463051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.463737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.464077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.464418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.466057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.467717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.492453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1228/2428] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:40.496321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.496479 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1201/2429] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:37:40.498285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.504489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c1f7-ddc2-446c-92f7-6d6bbecf30db None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.504773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c1f7-ddc2-446c-92f7-6d6bbecf30db None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.505071 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c1f7-ddc2-446c-92f7-6d6bbecf30db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.513780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeafc8-5a56-4d3c-a7f4-983d0821d166 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.519465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.520209 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.520951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.520951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.521636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.523489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.526646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5822c1f7-ddc2-446c-92f7-6d6bbecf30db None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:40.527902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5822c1f7-ddc2-446c-92f7-6d6bbecf30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.536491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1229/2430] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:37:40.537443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1202/2431] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:40.540771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.543816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822c1f7-ddc2-446c-92f7-6d6bbecf30db None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.552256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f91cdf-1a33-49c8-b468-16732e4c8525 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.555779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822c1f7-ddc2-446c-92f7-6d6bbecf30db None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.557301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.558083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.558432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.558779 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] PATH_INFO: `/v3/projects/6d4d61b4c94f435aadb688f6e0b47f25/users/8d4b575430184ee3a8b3c81cc81c84e4/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.560985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.561633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.561949 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.562217 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.562881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:37:40.563184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.564841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.581290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1203/2432] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:37:40.582484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.584019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4d61b4c94f435aadb688f6e0b47f25, user_id=8d4b575430184ee3a8b3c81cc81c84e4, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.586877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.593739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88430e2-5194-4db7-b2b9-78df71c196b4 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.600811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.601727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.602015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.602289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.602920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.604967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.608218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1230/2433] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:40 2022] PUT /identity/v3/projects/6d4d61b4c94f435aadb688f6e0b47f25/users/8d4b575430184ee3a8b3c81cc81c84e4/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:40.609937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.637453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1204/2434] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:37:40.640754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.684984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f53309-6606-4ccb-b9ff-7a1e27ca105e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.710995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a2b75-c57d-4d01-abea-6a73a1b52400 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.715810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.716451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.716733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.717010 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.717665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.721062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.742868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.743075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1205/2435] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:37:40.743481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.743809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.744038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.744681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.746608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.747263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.756517 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1231/2436] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:37:40.757240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a6181-dda1-4c0e-aa53-632d9de16d2b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.760413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.761984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.762578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.762859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.763135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.763777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.765965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.771078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3540bb-4f8d-43d0-bc1f-797f41d7ebfd admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.775726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512c528-9ff5-4eee-9a5a-badaa0bf9edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.776133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1206/2437] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:37:40.777603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c528-9ff5-4eee-9a5a-badaa0bf9edc admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.777966 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c528-9ff5-4eee-9a5a-badaa0bf9edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.778246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c528-9ff5-4eee-9a5a-badaa0bf9edc admin admin] PATH_INFO: `/v3/projects/6d4d61b4c94f435aadb688f6e0b47f25/users/8d4b575430184ee3a8b3c81cc81c84e4/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.779920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.780500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512c528-9ff5-4eee-9a5a-badaa0bf9edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4d61b4c94f435aadb688f6e0b47f25, user_id=8d4b575430184ee3a8b3c81cc81c84e4, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.783787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512c528-9ff5-4eee-9a5a-badaa0bf9edc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.790250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7375a-ecae-49c8-b163-1bc32e6473a9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.794110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1232/2438] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:40 2022] PUT /identity/v3/projects/6d4d61b4c94f435aadb688f6e0b47f25/users/8d4b575430184ee3a8b3c81cc81c84e4/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:40.804570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:40.806058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09426021-ef02-43d5-8fee-991661e17433 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.806340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09426021-ef02-43d5-8fee-991661e17433 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.806611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09426021-ef02-43d5-8fee-991661e17433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.837175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-09426021-ef02-43d5-8fee-991661e17433 None None] MFA Rules not processed for user `8d4b575430184ee3a8b3c81cc81c84e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:40.838500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-09426021-ef02-43d5-8fee-991661e17433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.849047 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.849676 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.849952 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.850225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.850858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.851019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:40.852456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.862349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1207/2439] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:37:40.866378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.875653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608362a-0ab6-4a05-96b1-9b690531d8f8 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:40.880125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace4c6b-78c5-4ae3-8735-feaf0636853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:40.880761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace4c6b-78c5-4ae3-8735-feaf0636853b admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.881025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace4c6b-78c5-4ae3-8735-feaf0636853b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.881318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace4c6b-78c5-4ae3-8735-feaf0636853b admin admin] PATH_INFO: `/v3/projects/22729064206f44009207e22ade61deda/users/5a8ddc5f3f844f1984f0b1e188d3662a/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.896379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace4c6b-78c5-4ae3-8735-feaf0636853b admin admin] RBAC: Authorizing `identity:create_grant(project_id=22729064206f44009207e22ade61deda, user_id=5a8ddc5f3f844f1984f0b1e188d3662a, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:40.899111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace4c6b-78c5-4ae3-8735-feaf0636853b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:40.909587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1233/2440] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:37:40.912765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1208/2441] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:37:40 2022] PUT /identity/v3/projects/22729064206f44009207e22ade61deda/users/5a8ddc5f3f844f1984f0b1e188d3662a/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:37:40.920794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09426021-ef02-43d5-8fee-991661e17433 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:40.922335 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-09426021-ef02-43d5-8fee-991661e17433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.922970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:40.923261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:40.923540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:40.951094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] MFA Rules not processed for user `5a8ddc5f3f844f1984f0b1e188d3662a`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:40.952926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:40.978232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09426021-ef02-43d5-8fee-991661e17433 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:41.018863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1209/2442] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:41.028126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98930ba1-2814-4005-ab71-edf8651051ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:41.028780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930ba1-2814-4005-ab71-edf8651051ba service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:41.029058 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930ba1-2814-4005-ab71-edf8651051ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:41.029359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930ba1-2814-4005-ab71-edf8651051ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:41.029704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:41.031092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:41.042884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac30c31-0018-4917-8aff-5c022e04f6ef None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:41.046986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:41.047579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:41.047882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:41.048137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:41.059161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:41.060671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:41.071694 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1210/2443] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:37:41.096612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98930ba1-2814-4005-ab71-edf8651051ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:41.098105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98930ba1-2814-4005-ab71-edf8651051ba service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:41.108341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1234/2444] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:41.204110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:41.217865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2c0f2-8a28-424c-b60c-67a9fa0c3231 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:41.227477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a6de8-909f-41b1-a37b-b25ebf359bc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:41.228805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a6de8-909f-41b1-a37b-b25ebf359bc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:41.229348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a6de8-909f-41b1-a37b-b25ebf359bc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:41.229867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a6de8-909f-41b1-a37b-b25ebf359bc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:41.234567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-476a6de8-909f-41b1-a37b-b25ebf359bc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:41.332860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a6de8-909f-41b1-a37b-b25ebf359bc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:41.335626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a6de8-909f-41b1-a37b-b25ebf359bc5 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:41.357377 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1211/2445] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:37:42.513245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:42.513793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:42.513793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:42.534600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] MFA Rules not processed for user `5a8ddc5f3f844f1984f0b1e188d3662a`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:42.535840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:42.543064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1235/2446] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:37:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:47.490567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:47.491039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:47.491151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:47.499277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:47.514512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:37:47.516224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:47.572912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1212/2447] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 10 12:37:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:37:47.585621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:47.587005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:47.598552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=100 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:47.658231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42782661-1b6f-4bf9-90ce-51704db59830 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:47.663284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385dfee-4e01-40c3-b1cd-691ca74eabc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:47.664360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385dfee-4e01-40c3-b1cd-691ca74eabc8 service swift] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:47.664360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385dfee-4e01-40c3-b1cd-691ca74eabc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:47.664534 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385dfee-4e01-40c3-b1cd-691ca74eabc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:47.672736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385dfee-4e01-40c3-b1cd-691ca74eabc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:47.674368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385dfee-4e01-40c3-b1cd-691ca74eabc8 service swift] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:47.682608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1236/2448] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:37:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:37:57.566147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=23 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:57.679229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=28 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:37:58.121999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:37:58.123390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:58.200672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5157ed-9a5a-4e00-840a-2e2ba6a53a5a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:37:58.206659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:37:58.207342 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:37:58.207624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:37:58.207900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:37:58.209680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:37:58.270088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:37:58.271846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:37:58.282269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1213/2449] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:02.103182 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89073f-c7ca-4a5c-8d24-c639562bbf58 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:02.103641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89073f-c7ca-4a5c-8d24-c639562bbf58 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:02.103733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89073f-c7ca-4a5c-8d24-c639562bbf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:02.123046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8e89073f-c7ca-4a5c-8d24-c639562bbf58 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:02.124213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8e89073f-c7ca-4a5c-8d24-c639562bbf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:02.131080 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1237/2450] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:38:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:02.142764 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:02.153634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462e8c9-1ab5-4d38-927c-4782d049824d service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:02.159380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:02.160051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:02.160354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:02.160624 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:02.168741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:02.170365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:02.178955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1214/2451] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:38:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:38:02.677164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33553a21-fe68-437a-a5d5-2173a533db56 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:02.677485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33553a21-fe68-437a-a5d5-2173a533db56 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:02.677755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33553a21-fe68-437a-a5d5-2173a533db56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:02.697157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-33553a21-fe68-437a-a5d5-2173a533db56 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:02.698366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-33553a21-fe68-437a-a5d5-2173a533db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:02.705647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1238/2452] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 10 12:38:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:02.718049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:02.728815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce36da-31c2-4896-8f30-94e35a94fb74 service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:02.734556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:02.735157 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:02.735434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:02.735723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:02.744019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:02.745689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:02.754339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1215/2453] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:38:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:38:02.974564 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60b17-d9af-4edb-956e-398a33991c09 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:02.974848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60b17-d9af-4edb-956e-398a33991c09 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:02.975126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60b17-d9af-4edb-956e-398a33991c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:02.996060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-31e60b17-d9af-4edb-956e-398a33991c09 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:02.997257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-31e60b17-d9af-4edb-956e-398a33991c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:03.005466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1239/2454] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:38:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:03.017241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:03.028175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bc548-53eb-4d86-9041-b56c8f319433 service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:03.036099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:03.036099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:03.036099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:03.036099 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:03.044906 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:03.047184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:03.055916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1216/2455] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:38:03.369147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad69954-7efc-4fa3-94f5-97f6995cabae None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:03.369647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad69954-7efc-4fa3-94f5-97f6995cabae None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:03.369739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad69954-7efc-4fa3-94f5-97f6995cabae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:03.388073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2ad69954-7efc-4fa3-94f5-97f6995cabae None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:03.389254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2ad69954-7efc-4fa3-94f5-97f6995cabae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:03.398930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1240/2456] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 10 12:38:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:03.412741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:03.423139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee649-9de9-459d-8629-0960c2acb27e service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:03.428233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bbe90-1cb2-4009-a07b-066803e8725d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:03.428829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bbe90-1cb2-4009-a07b-066803e8725d service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:03.429109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bbe90-1cb2-4009-a07b-066803e8725d service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:03.429412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bbe90-1cb2-4009-a07b-066803e8725d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:03.437760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bbe90-1cb2-4009-a07b-066803e8725d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:03.439336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bbe90-1cb2-4009-a07b-066803e8725d service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:03.447566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1217/2457] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:38:09.704880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad69954-7efc-4fa3-94f5-97f6995cabae None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:09.773147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad69954-7efc-4fa3-94f5-97f6995cabae None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:09.788098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:09.788778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:09.789059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:09.789379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] PATH_INFO: `/v3/users/5a8ddc5f3f844f1984f0b1e188d3662a` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:09.790592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8ddc5f3f844f1984f0b1e188d3662a)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:09.792667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:09.844214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8ddc5f3f844f1984f0b1e188d3662a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:09.851789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8ddc5f3f844f1984f0b1e188d3662a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:09.852527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8ddc5f3f844f1984f0b1e188d3662a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:09.853286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8ddc5f3f844f1984f0b1e188d3662a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:09.861595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8ddc5f3f844f1984f0b1e188d3662a'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:09.879498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1241/2458] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:09 2022] DELETE /identity/v3/users/5a8ddc5f3f844f1984f0b1e188d3662a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:09.883948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:09.884475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:09.884598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:09.903584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:09.904791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:09.945706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:09.965564 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1218/2459] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:09.975993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:09.977447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:10.034548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd384888-e1db-4b8f-9398-fc2b9120fdcb admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.038804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139702d-37c5-4161-90f7-afba9a257132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.039426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139702d-37c5-4161-90f7-afba9a257132 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.039689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139702d-37c5-4161-90f7-afba9a257132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.039956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139702d-37c5-4161-90f7-afba9a257132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.047094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:10.049586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139702d-37c5-4161-90f7-afba9a257132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.051144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139702d-37c5-4161-90f7-afba9a257132 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.062051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1242/2460] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:10.226301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.228510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:10.243588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba827ca1-f093-4125-b59c-ff1b5a7be7dd None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.249429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.250258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.250614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.251257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] PATH_INFO: `/v3/projects/22729064206f44009207e22ade61deda` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.253332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22729064206f44009207e22ade61deda)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.256297 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.308314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22729064206f44009207e22ade61deda', 'request_id': 'req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:10.314737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22729064206f44009207e22ade61deda', 'request_id': 'req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:10.323331 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22729064206f44009207e22ade61deda', 'request_id': 'req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:10.339250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] The token cache is being invalidate because project 22729064206f44009207e22ade61deda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:10.339698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:10.340568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f13b795d-ea0b-4915-a647-9d1deb5d5b2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:10.342527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1219/2461] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:10 2022] DELETE /identity/v3/projects/22729064206f44009207e22ade61deda => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:10.348122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2752f5d-914d-4a38-b6a2-4ede7578e525 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.348476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2752f5d-914d-4a38-b6a2-4ede7578e525 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.348823 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2752f5d-914d-4a38-b6a2-4ede7578e525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.368342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e2752f5d-914d-4a38-b6a2-4ede7578e525 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:10.369587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e2752f5d-914d-4a38-b6a2-4ede7578e525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:10.437678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1243/2462] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:10.445937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0dbe-d64d-44e3-8793-b46a02b8ba4a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.446231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0dbe-d64d-44e3-8793-b46a02b8ba4a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.446538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0dbe-d64d-44e3-8793-b46a02b8ba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.466528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-eafd0dbe-d64d-44e3-8793-b46a02b8ba4a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:10.468209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-eafd0dbe-d64d-44e3-8793-b46a02b8ba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:10.478049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1220/2463] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:10.482370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2752f5d-914d-4a38-b6a2-4ede7578e525 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.492576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2752f5d-914d-4a38-b6a2-4ede7578e525 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.497395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.498129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.498471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.498778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.499431 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:10.499736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.501444 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.513063 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1244/2464] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:38:10.517517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0dbe-d64d-44e3-8793-b46a02b8ba4a None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.526669 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0dbe-d64d-44e3-8793-b46a02b8ba4a None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.531765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.532352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.532626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.532890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.533535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.535695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.565141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1221/2465] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:10 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:10.569099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.579934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe10e8-e32d-4993-9401-df55ee860863 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.641929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.642467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.642750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.643026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.643661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.645629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.668381 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1245/2466] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:10 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:10.673160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.683188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7436b-e3c6-4860-a1a6-413036ec070f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.687758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.688354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.688632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.688899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.689569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.691193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.701483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1222/2467] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:10.706307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.716447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c183b-8180-40f7-86c2-e62344a7527c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.721981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e956-c875-4788-a303-d00174719791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.722577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e956-c875-4788-a303-d00174719791 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.722855 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e956-c875-4788-a303-d00174719791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.723126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e956-c875-4788-a303-d00174719791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.723767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e956-c875-4788-a303-d00174719791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.725556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e956-c875-4788-a303-d00174719791 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.737443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1246/2468] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:10.741458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.751787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ea07a-4ecc-4631-ad9e-f04a276f4ea7 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.756619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.757305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.757614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.757892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] PATH_INFO: `/v3/projects/65ece09339af406b98e493fa33f2b9c0/users/94df22112d134130a678f91cae49adef/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.773574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ece09339af406b98e493fa33f2b9c0, user_id=94df22112d134130a678f91cae49adef, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.776230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.788815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1223/2469] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:10 2022] PUT /identity/v3/projects/65ece09339af406b98e493fa33f2b9c0/users/94df22112d134130a678f91cae49adef/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:10.793647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc710f5-6980-4855-ada1-d971943ce78f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.794130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc710f5-6980-4855-ada1-d971943ce78f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.794443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc710f5-6980-4855-ada1-d971943ce78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.815470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6cc710f5-6980-4855-ada1-d971943ce78f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:10.816704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6cc710f5-6980-4855-ada1-d971943ce78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:10.880983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1247/2470] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:10.891397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:10.893291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:10.950770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47fee7-ef07-4497-9cff-755a00ccb125 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:10.954871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:10.955471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:10.955749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:10.956352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:10.971285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:10.971285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:10.981112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1224/2471] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:15.326412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69401acb-9d37-4864-8ba7-2265740773b7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:15.327507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69401acb-9d37-4864-8ba7-2265740773b7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:15.327507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69401acb-9d37-4864-8ba7-2265740773b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:15.334555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:15.360241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-69401acb-9d37-4864-8ba7-2265740773b7 None None] MFA Rules not processed for user `94df22112d134130a678f91cae49adef`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:15.361487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-69401acb-9d37-4864-8ba7-2265740773b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:15.429597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1248/2472] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:15.434439 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:15.444689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b71f5-2571-498b-a4b0-b25bd8fef463 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:15.449333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:15.449939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:15.450216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:15.450484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:15.451115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:15.453578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:15.458571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:15.486826 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1225/2473] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:15 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:15.488241 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69401acb-9d37-4864-8ba7-2265740773b7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:15.499034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69401acb-9d37-4864-8ba7-2265740773b7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:15.557608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:15.558300 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:15.558645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:15.558994 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:15.559687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:15.562436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:15.584733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1249/2474] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:15 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:15.588842 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:15.599358 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b11d-99f2-4fc0-8c9d-e20702b4178f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:15.603946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:15.604538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:15.604812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:15.605085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:15.605766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:15.607364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:15.617985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1226/2475] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:15.622094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:15.632614 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df51f0b-e6dd-4e97-a62a-d53a59764d47 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:15.637177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:15.637853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:15.638134 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:15.638407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] PATH_INFO: `/v3/projects/027a9b17f5e844bc976eaf5db7f51e25/users/85190e1676ed4086b802d78875c8f7cf/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:15.653440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=027a9b17f5e844bc976eaf5db7f51e25, user_id=85190e1676ed4086b802d78875c8f7cf, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:15.656385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:15.669610 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1250/2476] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:15 2022] PUT /identity/v3/projects/027a9b17f5e844bc976eaf5db7f51e25/users/85190e1676ed4086b802d78875c8f7cf/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:15.673568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:15.683487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388445-d387-4269-8619-a3820aadbefd admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:15.741614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5291f63-9946-4a42-b98f-997bae066652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:15.742386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5291f63-9946-4a42-b98f-997bae066652 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:15.742665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5291f63-9946-4a42-b98f-997bae066652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:15.742959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5291f63-9946-4a42-b98f-997bae066652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:15.743614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5291f63-9946-4a42-b98f-997bae066652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:15.745252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5291f63-9946-4a42-b98f-997bae066652 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:15.756707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1227/2477] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:15.760776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:15.771979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c126e1-a4e0-4ffc-80e1-16f34c179715 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:15.776572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:15.777379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:15.777720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:15.778059 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] PATH_INFO: `/v3/projects/027a9b17f5e844bc976eaf5db7f51e25/users/85190e1676ed4086b802d78875c8f7cf/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:15.780054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=027a9b17f5e844bc976eaf5db7f51e25, user_id=85190e1676ed4086b802d78875c8f7cf, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:15.782828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:15.792917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1251/2478] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:15 2022] PUT /identity/v3/projects/027a9b17f5e844bc976eaf5db7f51e25/users/85190e1676ed4086b802d78875c8f7cf/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:20.268128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c57b7-9519-4309-a232-338442014908 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:20.268986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c57b7-9519-4309-a232-338442014908 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:20.269787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c57b7-9519-4309-a232-338442014908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:20.318511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a30c57b7-9519-4309-a232-338442014908 None None] MFA Rules not processed for user `85190e1676ed4086b802d78875c8f7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:20.319745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a30c57b7-9519-4309-a232-338442014908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:20.421905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1228/2479] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:20.435596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:20.438530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:20.505216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:20.550897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf87cf6-e75b-41f9-8728-6791254a7b6a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:20.558808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:20.560325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:20.560806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:20.562281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:20.640483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:20.656925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:20.694090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1252/2480] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:22.024136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4028fb-0c51-4e79-a0fe-3412f75fcd2c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:22.024926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4028fb-0c51-4e79-a0fe-3412f75fcd2c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:22.025196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4028fb-0c51-4e79-a0fe-3412f75fcd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:22.055327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-db4028fb-0c51-4e79-a0fe-3412f75fcd2c None None] MFA Rules not processed for user `94df22112d134130a678f91cae49adef`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:22.056647 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-db4028fb-0c51-4e79-a0fe-3412f75fcd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:22.064690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1229/2481] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:22.891276 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:22.892745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:22.964615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6875c-7ff8-4a70-9094-bf5a41907e12 service nova] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:22.969793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:22.970533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:22.971282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:22.971620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] PATH_INFO: `/v3/users/c7b456384631453989464d7114b0d4e2` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:22.972834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7b456384631453989464d7114b0d4e2)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:22.974869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:23.032037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7b456384631453989464d7114b0d4e2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.039828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7b456384631453989464d7114b0d4e2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.040516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7b456384631453989464d7114b0d4e2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.041142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7b456384631453989464d7114b0d4e2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.049249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7b456384631453989464d7114b0d4e2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.072661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:23.087202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1253/2482] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:22 2022] DELETE /identity/v3/users/c7b456384631453989464d7114b0d4e2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:23.092313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:23.092837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:23.092921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:23.125483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:23.125483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:23.176915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:23.196277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1230/2483] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:23.207670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:23.209178 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:23.274607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2816a-135d-46e8-81e0-524b87810b1c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:23.278899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:23.279549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:23.280179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:23.280858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:23.293453 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:23.297654 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:23.309637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1254/2484] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:23.634726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:23.636308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:23.651262 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159ed0e-aaeb-4d00-b6e8-7ec7515d1525 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:23.661118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:23.661806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:23.669685 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:23.671253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] PATH_INFO: `/v3/projects/e9319a14d6ae414d9fb614141dc410b9` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:23.673306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9319a14d6ae414d9fb614141dc410b9)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:23.675400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:23.755696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9319a14d6ae414d9fb614141dc410b9', 'request_id': 'req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.764974 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9319a14d6ae414d9fb614141dc410b9', 'request_id': 'req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.773499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9319a14d6ae414d9fb614141dc410b9', 'request_id': 'req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.785936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] The token cache is being invalidate because project e9319a14d6ae414d9fb614141dc410b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:23.786227 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.786888 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.788387 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1231/2485] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:23 2022] DELETE /identity/v3/projects/e9319a14d6ae414d9fb614141dc410b9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:23.792327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:23.803192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffceed-e97d-4829-8423-5fd912a36440 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:23.878627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:23.879266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:23.879540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:23.879803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] PATH_INFO: `/v3/users/8d4b575430184ee3a8b3c81cc81c84e4` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:23.880935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4b575430184ee3a8b3c81cc81c84e4)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:23.882903 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:23.934689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4b575430184ee3a8b3c81cc81c84e4'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.942396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4b575430184ee3a8b3c81cc81c84e4'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.943046 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4b575430184ee3a8b3c81cc81c84e4'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.943700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4b575430184ee3a8b3c81cc81c84e4'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.951581 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bc147bfe-6b4d-46da-b946-985db325299d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4b575430184ee3a8b3c81cc81c84e4'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:23.970179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1255/2486] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:23 2022] DELETE /identity/v3/users/8d4b575430184ee3a8b3c81cc81c84e4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:24.679817 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:24.681914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:24.706207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb7e46-5a1d-4aa6-86aa-f3e7e2396826 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:24.775213 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:24.776189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:24.776472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:24.776756 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] PATH_INFO: `/v3/projects/6d4d61b4c94f435aadb688f6e0b47f25` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:24.778302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d4d61b4c94f435aadb688f6e0b47f25)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:24.780854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:24.839376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d4d61b4c94f435aadb688f6e0b47f25', 'request_id': 'req-243fed14-83bf-41b5-a47e-d46ab8cf1439'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:24.846461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d4d61b4c94f435aadb688f6e0b47f25', 'request_id': 'req-243fed14-83bf-41b5-a47e-d46ab8cf1439'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:24.854444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d4d61b4c94f435aadb688f6e0b47f25', 'request_id': 'req-243fed14-83bf-41b5-a47e-d46ab8cf1439'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:24.866060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] The token cache is being invalidate because project 6d4d61b4c94f435aadb688f6e0b47f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:24.866349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:24.866985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-243fed14-83bf-41b5-a47e-d46ab8cf1439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:24.868674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1232/2487] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:24 2022] DELETE /identity/v3/projects/6d4d61b4c94f435aadb688f6e0b47f25 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:24.874893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1ca51-fe77-4b92-a2b9-dab1fde86d48 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:24.875211 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1ca51-fe77-4b92-a2b9-dab1fde86d48 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:24.875487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1ca51-fe77-4b92-a2b9-dab1fde86d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:24.896840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fbd1ca51-fe77-4b92-a2b9-dab1fde86d48 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:24.898068 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fbd1ca51-fe77-4b92-a2b9-dab1fde86d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:24.915590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:24.968087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1256/2488] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:24.976521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e9c9-61e5-4fb0-b71a-3d086aa78066 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:24.976692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e9c9-61e5-4fb0-b71a-3d086aa78066 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:24.977039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e9c9-61e5-4fb0-b71a-3d086aa78066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:24.987492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:25.002325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ab58e9c9-61e5-4fb0-b71a-3d086aa78066 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:25.004203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ab58e9c9-61e5-4fb0-b71a-3d086aa78066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:25.013890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1233/2489] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:25.018306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1ca51-fe77-4b92-a2b9-dab1fde86d48 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.029228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1ca51-fe77-4b92-a2b9-dab1fde86d48 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.038500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.039608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.039897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.040208 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.041018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:25.041232 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.046077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.059609 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1257/2490] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:25.064102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e9c9-61e5-4fb0-b71a-3d086aa78066 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.073258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e9c9-61e5-4fb0-b71a-3d086aa78066 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.077861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.078452 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.078730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.079007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.079635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.081570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.114979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1234/2491] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:25 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:25.120639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.137806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92592-37e7-4458-84ff-75a1044b41c1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.204720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.205366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.205677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.205951 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.206580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.208526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.233529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1258/2492] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:25 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:25.234753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.244844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a904984-bb2b-417c-bba8-f51dde32777a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.250034 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.250621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.250895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.251161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.251794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.253420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.263992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1235/2493] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:25.267899 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.277636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f560c-969a-4561-9856-4d02ff2bacce admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.282257 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cd0f2-b877-4df1-97eb-8d0408464f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.283010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cd0f2-b877-4df1-97eb-8d0408464f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.283293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cd0f2-b877-4df1-97eb-8d0408464f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.283587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cd0f2-b877-4df1-97eb-8d0408464f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.284225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cd0f2-b877-4df1-97eb-8d0408464f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.285853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cd0f2-b877-4df1-97eb-8d0408464f5e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.296858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1259/2494] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:25.301121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.311722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b189c-3fc2-442e-afda-5afabc1cdb6b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.316319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.316965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.317241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.317552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] PATH_INFO: `/v3/projects/c39fab90c7024325a0e908386984135b/users/c649d380c30a48a08571ec37910348e2/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.332914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c39fab90c7024325a0e908386984135b, user_id=c649d380c30a48a08571ec37910348e2, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.335637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.351381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1236/2495] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:25 2022] PUT /identity/v3/projects/c39fab90c7024325a0e908386984135b/users/c649d380c30a48a08571ec37910348e2/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:25.363397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f026b0d-e1ea-4a7f-a0a9-7be578699a89 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.363713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f026b0d-e1ea-4a7f-a0a9-7be578699a89 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.363990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f026b0d-e1ea-4a7f-a0a9-7be578699a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.396378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-3f026b0d-e1ea-4a7f-a0a9-7be578699a89 None None] MFA Rules not processed for user `c649d380c30a48a08571ec37910348e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:25.397774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3f026b0d-e1ea-4a7f-a0a9-7be578699a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:25.474842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1260/2496] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:25.479333 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.543296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046d4ec-3391-4852-91cf-d08eb7b0825f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.548599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.549233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.549536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.549808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.550430 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.552360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.582229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1237/2497] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:25 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:25.586450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f026b0d-e1ea-4a7f-a0a9-7be578699a89 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.600455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f026b0d-e1ea-4a7f-a0a9-7be578699a89 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.613816 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:25.666714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.667344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.667625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.667895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.668553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.671101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.705554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1261/2498] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:25 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:25.708732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.722170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63df93f-a64b-44b3-9542-646b4653cd2a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.728528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.729428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.729713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.729985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.730633 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.732354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.744049 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1238/2499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:25.747985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.758636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3c522-e980-496f-8a8b-ae6f1a75ffb6 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.763263 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.763911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.764225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.764498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] PATH_INFO: `/v3/projects/3645666a6f8242808b7a5ddfe48d42a8/users/221acea551a940cab5f7d9598f6de5aa/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.779293 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3645666a6f8242808b7a5ddfe48d42a8, user_id=221acea551a940cab5f7d9598f6de5aa, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.782029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.796199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1262/2500] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:25 2022] PUT /identity/v3/projects/3645666a6f8242808b7a5ddfe48d42a8/users/221acea551a940cab5f7d9598f6de5aa/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:25.800023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.815503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894a26d-1b26-4fea-b52b-d31a31407282 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.880937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bc4e8-6901-4c51-9c3e-fc32effcb466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.881591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bc4e8-6901-4c51-9c3e-fc32effcb466 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.881870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bc4e8-6901-4c51-9c3e-fc32effcb466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.882159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bc4e8-6901-4c51-9c3e-fc32effcb466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.882791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2bc4e8-6901-4c51-9c3e-fc32effcb466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.884487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2bc4e8-6901-4c51-9c3e-fc32effcb466 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.897382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1239/2501] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:25.900532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:25.915742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99897f-c7d1-4940-adec-3d680da4a96c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:25.920976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:25.921811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.922138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.922486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] PATH_INFO: `/v3/projects/3645666a6f8242808b7a5ddfe48d42a8/users/221acea551a940cab5f7d9598f6de5aa/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.924627 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3645666a6f8242808b7a5ddfe48d42a8, user_id=221acea551a940cab5f7d9598f6de5aa, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:25.927387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:25.939000 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1263/2502] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:25 2022] PUT /identity/v3/projects/3645666a6f8242808b7a5ddfe48d42a8/users/221acea551a940cab5f7d9598f6de5aa/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:25.949474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca477-e850-4893-a95e-56e4582a544c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:25.950039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca477-e850-4893-a95e-56e4582a544c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:25.950560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca477-e850-4893-a95e-56e4582a544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:25.980766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:25.997476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2bdca477-e850-4893-a95e-56e4582a544c None None] MFA Rules not processed for user `221acea551a940cab5f7d9598f6de5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:25.999090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2bdca477-e850-4893-a95e-56e4582a544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:26.083224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1240/2503] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:25 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:26.095192 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:26.098558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:26.166685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd805311-17a6-45c4-8949-ef9ed6052e9a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:26.170349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:26.171001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:26.171279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:26.171570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:26.181216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:26.182720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:26.192962 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1264/2504] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:26.283863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca477-e850-4893-a95e-56e4582a544c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:26.293451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca477-e850-4893-a95e-56e4582a544c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:26.296976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:26.297600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:26.297879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:26.298145 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:26.365090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:26.367002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:26.377111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1241/2505] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:35.138902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:35.140911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:35.197207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16000b-aa6a-444b-9aaa-7ce0ac3428bb service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:35.201247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:35.202697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:35.203094 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:35.203505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:35.205760 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:35.235923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:35.273703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:35.275150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:35.284618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1265/2506] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:36.372486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=58 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:37.294485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:37.304578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624444-0fbc-410f-84b8-d9a7652d79eb service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:37.308188 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:37.308783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:37.309060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:37.309362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:37.311751 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:37.326032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:37.327510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:37.338631 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1242/2507] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:38.628165 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:38.630122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:38.652217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee99a1-44c0-468c-ba03-da34d0918f99 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:38.658800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:38.659715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:38.660195 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:38.661062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] PATH_INFO: `/v3/users/e6842565de354b0c85b04940d57c7432` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:38.662960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6842565de354b0c85b04940d57c7432)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:38.665886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:38.762346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6842565de354b0c85b04940d57c7432'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:38.773575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6842565de354b0c85b04940d57c7432'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:38.778217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6842565de354b0c85b04940d57c7432'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:38.778854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6842565de354b0c85b04940d57c7432'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:38.788908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4784214f-e50c-4ffa-9551-b83f33dc68e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6842565de354b0c85b04940d57c7432'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:38.817639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1266/2508] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:38 2022] DELETE /identity/v3/users/e6842565de354b0c85b04940d57c7432 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:39.038975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.062463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896646c-77e2-4a73-883d-0429fdb911b9 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:39.129057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:39.129843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.130122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.130396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] PATH_INFO: `/v3/projects/82c24c64f4c34921837eceb32e24a287` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.131568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c24c64f4c34921837eceb32e24a287)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:39.133542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:39.189167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c24c64f4c34921837eceb32e24a287', 'request_id': 'req-3070ab02-3824-4687-b11a-fe8ad9ab65de'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:39.197243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c24c64f4c34921837eceb32e24a287', 'request_id': 'req-3070ab02-3824-4687-b11a-fe8ad9ab65de'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:39.209823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c24c64f4c34921837eceb32e24a287', 'request_id': 'req-3070ab02-3824-4687-b11a-fe8ad9ab65de'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:39.226118 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] The token cache is being invalidate because project 82c24c64f4c34921837eceb32e24a287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:39.226674 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:39.227612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-3070ab02-3824-4687-b11a-fe8ad9ab65de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:39.229688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1243/2509] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:39 2022] DELETE /identity/v3/projects/82c24c64f4c34921837eceb32e24a287 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:39.234922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b648e-8d76-4bf0-953a-7b3eafd13618 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.235254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b648e-8d76-4bf0-953a-7b3eafd13618 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.235533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b648e-8d76-4bf0-953a-7b3eafd13618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.256808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-752b648e-8d76-4bf0-953a-7b3eafd13618 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:39.258312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-752b648e-8d76-4bf0-953a-7b3eafd13618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:39.328125 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1267/2510] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:39.336813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65fd31-d01a-479b-b053-3dfb15748194 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.337130 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65fd31-d01a-479b-b053-3dfb15748194 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.337580 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65fd31-d01a-479b-b053-3dfb15748194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.361522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-fc65fd31-d01a-479b-b053-3dfb15748194 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:39.362940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-fc65fd31-d01a-479b-b053-3dfb15748194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:39.374677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1244/2511] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:39.376042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b648e-8d76-4bf0-953a-7b3eafd13618 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.386089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b648e-8d76-4bf0-953a-7b3eafd13618 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:39.391333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:39.391942 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.392224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.392498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.393152 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:39.393486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:39.395105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:39.410438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1268/2512] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:38:39.415026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65fd31-d01a-479b-b053-3dfb15748194 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.425604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65fd31-d01a-479b-b053-3dfb15748194 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:39.431273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:39.431931 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.432215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.432947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.433294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:39.435237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:39.470022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1245/2513] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:39 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:39.474620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.485364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb8f8c-b96b-4bfe-a780-28f3d967687d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:39.516375 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:39.567433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:39.568139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.568558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.568854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.569510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:39.571464 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:39.633649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1269/2514] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:39 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:39.635354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.650095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84eb1bc-f5cd-4f65-b751-5f595a261d67 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:39.656098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:39.656710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.656983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.657254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.657955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:39.659567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:39.676694 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.681352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1246/2515] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:39.689410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087e837-2a87-4e8a-855e-989be22d0fc5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:39.694127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15ed6d-2419-4371-baa7-78d7ec32db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:39.695039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ed6d-2419-4371-baa7-78d7ec32db5f admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.695325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ed6d-2419-4371-baa7-78d7ec32db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.695599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ed6d-2419-4371-baa7-78d7ec32db5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.696249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15ed6d-2419-4371-baa7-78d7ec32db5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:39.697890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15ed6d-2419-4371-baa7-78d7ec32db5f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:39.710429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1270/2516] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:39.715149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.728030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa8b6-ee72-4950-a8c1-ec16c07edc2b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:39.734022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:39.734793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.735141 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.735529 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] PATH_INFO: `/v3/projects/e23b799cef7e44ffa8fd6278eec6396c/users/cd0238d8b217440e84327957c4b9ade1/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.755839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23b799cef7e44ffa8fd6278eec6396c, user_id=cd0238d8b217440e84327957c4b9ade1, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:39.758774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:39.781343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1247/2517] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:39 2022] PUT /identity/v3/projects/e23b799cef7e44ffa8fd6278eec6396c/users/cd0238d8b217440e84327957c4b9ade1/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:39.788089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:39.789618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:39.789778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:39.825952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] MFA Rules not processed for user `cd0238d8b217440e84327957c4b9ade1`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:39.828306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:39.903115 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1271/2518] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:39.914404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:39.916197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:39.930603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:40.003745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17a8c4-a6d1-4c5b-94d1-435778f6f1fe admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:40.010663 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:40.011225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:40.011510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:40.011783 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:40.022380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:40.023989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:40.033448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1248/2519] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:40.946913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:40.948414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:41.031571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ea2ef-1ae5-42df-8e81-151ec66c563a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:41.037959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:41.039143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.039143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.039421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] PATH_INFO: `/v3/users/a78bd1aed509474b965592c478cb1c0c` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.040896 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78bd1aed509474b965592c478cb1c0c)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:41.043193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:41.088159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:41.107538 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78bd1aed509474b965592c478cb1c0c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.115971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78bd1aed509474b965592c478cb1c0c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.116657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78bd1aed509474b965592c478cb1c0c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.117331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78bd1aed509474b965592c478cb1c0c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.125845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02198fbb-a088-4e40-bea6-019a89cb5074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78bd1aed509474b965592c478cb1c0c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.147541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1272/2520] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:40 2022] DELETE /identity/v3/users/a78bd1aed509474b965592c478cb1c0c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:41.337926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:41.351376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f7c2b-c456-42b0-b263-057c81441046 service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:41.422267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:41.422905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.423242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.423524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] PATH_INFO: `/v3/projects/4eb5c9a17d7341d3a3dd60ef2b7a381d` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.424811 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb5c9a17d7341d3a3dd60ef2b7a381d)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:41.426827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:41.478752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb5c9a17d7341d3a3dd60ef2b7a381d', 'request_id': 'req-5024e620-32f2-4047-b223-cc6639962ec7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.484838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb5c9a17d7341d3a3dd60ef2b7a381d', 'request_id': 'req-5024e620-32f2-4047-b223-cc6639962ec7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.492593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb5c9a17d7341d3a3dd60ef2b7a381d', 'request_id': 'req-5024e620-32f2-4047-b223-cc6639962ec7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.506043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] The token cache is being invalidate because project 4eb5c9a17d7341d3a3dd60ef2b7a381d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:41.506335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.506978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5024e620-32f2-4047-b223-cc6639962ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:41.508643 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1249/2521] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:41 2022] DELETE /identity/v3/projects/4eb5c9a17d7341d3a3dd60ef2b7a381d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:41.513234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2aa4c-273a-46a0-b40c-f734e2edae17 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.513556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2aa4c-273a-46a0-b40c-f734e2edae17 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.513863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2aa4c-273a-46a0-b40c-f734e2edae17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.533333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-43b2aa4c-273a-46a0-b40c-f734e2edae17 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:41.534549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-43b2aa4c-273a-46a0-b40c-f734e2edae17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:41.603788 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1273/2522] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:41.612890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8a1cd-c8eb-44cd-8c8b-d8b5cfd4376c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.613243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8a1cd-c8eb-44cd-8c8b-d8b5cfd4376c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.613538 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8a1cd-c8eb-44cd-8c8b-d8b5cfd4376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.636284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b7f8a1cd-c8eb-44cd-8c8b-d8b5cfd4376c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:41.637554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b7f8a1cd-c8eb-44cd-8c8b-d8b5cfd4376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:41.646678 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1250/2523] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:41.650603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2aa4c-273a-46a0-b40c-f734e2edae17 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:41.666119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2aa4c-273a-46a0-b40c-f734e2edae17 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:41.673976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:41.674604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.674885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.675317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.675981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:41.676287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:41.678218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:41.697647 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1274/2524] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:41.699621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8a1cd-c8eb-44cd-8c8b-d8b5cfd4376c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:41.712460 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8a1cd-c8eb-44cd-8c8b-d8b5cfd4376c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:41.717891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:41.718505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.718792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.721184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.721868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:41.723834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:41.761526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:41.771648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1251/2525] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:41 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:41.775913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:41.794535 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a9f00-6d62-4444-a10a-4f7d33efa7df admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:41.882176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:41.882940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.883259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.883630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.884362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:41.886645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:41.927352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1275/2526] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:41 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:41.929030 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:41.940597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b331-991d-4b68-9ebc-6b26bdaae7c6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:41.946336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:41.947161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.947445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.947724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.948374 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:41.950349 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:41.961399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1252/2527] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:41.965686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:41.978369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0a731-2cd3-4b4a-8f53-2923175c1650 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:41.985113 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:41.985775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:41.986116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:41.986513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:41.987233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:41.988986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.002518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1276/2528] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:42.005226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.005741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.007395 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.020042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ad0c8-b209-4e3d-b7e1-702c4b465a4b admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.024724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce0ba9-9a21-4566-bc3f-ebdee8ca96ce admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.028044 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38881a-36b3-405f-973f-b85712269ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.030565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.030735 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38881a-36b3-405f-973f-b85712269ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.031047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38881a-36b3-405f-973f-b85712269ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.031395 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38881a-36b3-405f-973f-b85712269ac1 admin admin] PATH_INFO: `/v3/projects/20a373b8272441549f04aad0f0677667/users/7cb96c148d3c4ec58b21eefd647b512c/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.031547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.031913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.032250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] PATH_INFO: `/v3/users/c649d380c30a48a08571ec37910348e2` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.033709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c649d380c30a48a08571ec37910348e2)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.036613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.049314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:42.054415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38881a-36b3-405f-973f-b85712269ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a373b8272441549f04aad0f0677667, user_id=7cb96c148d3c4ec58b21eefd647b512c, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.057763 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38881a-36b3-405f-973f-b85712269ac1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.077486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1277/2529] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:42 2022] PUT /identity/v3/projects/20a373b8272441549f04aad0f0677667/users/7cb96c148d3c4ec58b21eefd647b512c/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:42.088989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.089997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.090280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.105026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c649d380c30a48a08571ec37910348e2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.115410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c649d380c30a48a08571ec37910348e2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.123340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] MFA Rules not processed for user `7cb96c148d3c4ec58b21eefd647b512c`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:42.124594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.125016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c649d380c30a48a08571ec37910348e2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.125786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c649d380c30a48a08571ec37910348e2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.126528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9bd79b4e-6121-4840-98b5-e4e44a9f258a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c649d380c30a48a08571ec37910348e2'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.153873 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1253/2530] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:42 2022] DELETE /identity/v3/users/c649d380c30a48a08571ec37910348e2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:42.159701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.160012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.160304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.183085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:42.184453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.202998 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1278/2531] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:42.210328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.211745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.257311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1254/2532] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:42.267296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.268849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.293605 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fc913-48aa-44da-9284-ef0d38e859b6 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.299197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.299808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.300137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.300408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.301036 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.302988 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.333298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1279/2533] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:42 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:42.335496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.344436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589a19b-9fe3-4604-9a09-48e5033d787f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.356926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:42.399035 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.400251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.400547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.400824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.403978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dce9fc-fbb0-40a0-a695-3a47f9f573f8 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.411663 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.412370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.412620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.412842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.413089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.413523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.414741 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.415638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.427296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:42.428226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1255/2534] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:42.439883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1280/2535] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:42 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:42.442896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.454245 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30b505-ea24-4c55-81c8-fc7e8862a6be service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.459148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.459595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.459868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.460325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.460860 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.462810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.476215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1256/2536] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:42.480285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.491414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c4c69-c1c4-4cf9-8aba-5388994294aa admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.496629 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.497432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.497711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.498005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] PATH_INFO: `/v3/projects/dba07646f5d243219b808a65fa0d4c4f/users/fc920003220f417ea5bb59cb80e9d286/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.514732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba07646f5d243219b808a65fa0d4c4f, user_id=fc920003220f417ea5bb59cb80e9d286, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.517547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.531956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1281/2537] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:42 2022] PUT /identity/v3/projects/dba07646f5d243219b808a65fa0d4c4f/users/fc920003220f417ea5bb59cb80e9d286/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:42.535928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.546346 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ce846-ee35-4f97-92f2-fa20c1372905 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.619052 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ef493-a3fe-4d63-8375-94755f49752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.619752 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ef493-a3fe-4d63-8375-94755f49752e admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.620033 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ef493-a3fe-4d63-8375-94755f49752e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.620306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ef493-a3fe-4d63-8375-94755f49752e admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.620961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ef493-a3fe-4d63-8375-94755f49752e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.622640 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ef493-a3fe-4d63-8375-94755f49752e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.633611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1257/2538] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:42.637844 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.649022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca10b4-794c-46ec-892f-bad223b208e2 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.653684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.654339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.654644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.654943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] PATH_INFO: `/v3/projects/dba07646f5d243219b808a65fa0d4c4f/users/fc920003220f417ea5bb59cb80e9d286/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.656980 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba07646f5d243219b808a65fa0d4c4f, user_id=fc920003220f417ea5bb59cb80e9d286, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.659724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.670052 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1282/2539] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:42 2022] PUT /identity/v3/projects/dba07646f5d243219b808a65fa0d4c4f/users/fc920003220f417ea5bb59cb80e9d286/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:42.681609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.681895 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.682169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.716323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] MFA Rules not processed for user `fc920003220f417ea5bb59cb80e9d286`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:42.717579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.721504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.723396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.817284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1258/2540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:42.831594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.833284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:42.849549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff2d7-7076-409f-ad28-bd4dd4a7a098 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.873871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.873871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.873871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.873871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] PATH_INFO: `/v3/projects/c39fab90c7024325a0e908386984135b` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.874295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c39fab90c7024325a0e908386984135b)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.874295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.893574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:42.926202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642a963-1af1-4b76-990d-06946f55c0ea None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:42.930136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:42.930908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:42.931363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:42.931758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:42.942865 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:42.944579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:42.945870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c39fab90c7024325a0e908386984135b', 'request_id': 'req-616efc29-2539-4542-936d-9325a2e46af6'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.953652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c39fab90c7024325a0e908386984135b', 'request_id': 'req-616efc29-2539-4542-936d-9325a2e46af6'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.956718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1259/2541] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:42.963344 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c39fab90c7024325a0e908386984135b', 'request_id': 'req-616efc29-2539-4542-936d-9325a2e46af6'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.976774 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] The token cache is being invalidate because project c39fab90c7024325a0e908386984135b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:42.977405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.977963 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:42.979778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1283/2542] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:42 2022] DELETE /identity/v3/projects/c39fab90c7024325a0e908386984135b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:42.984767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:42.997757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067472dc-fff4-4918-bde0-7a443db34dce service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:43.063690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:43.064618 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.065105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.065547 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] PATH_INFO: `/v3/users/221acea551a940cab5f7d9598f6de5aa` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.067410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=221acea551a940cab5f7d9598f6de5aa)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:43.070216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:43.147546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221acea551a940cab5f7d9598f6de5aa'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.154806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:43.156294 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:43.157477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221acea551a940cab5f7d9598f6de5aa'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.167726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221acea551a940cab5f7d9598f6de5aa'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.168456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221acea551a940cab5f7d9598f6de5aa'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.169121 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-60f927aa-bcbd-48ad-ab42-80e715d6a760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221acea551a940cab5f7d9598f6de5aa'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.177971 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616efc29-2539-4542-936d-9325a2e46af6 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:43.189236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:43.190343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.190782 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.191288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] PATH_INFO: `/v3/users/cd0238d8b217440e84327957c4b9ade1` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.192747 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd0238d8b217440e84327957c4b9ade1)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:43.194907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:43.196516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1260/2543] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:42 2022] DELETE /identity/v3/users/221acea551a940cab5f7d9598f6de5aa => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:43.263576 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:43.266988 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd0238d8b217440e84327957c4b9ade1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.278200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd0238d8b217440e84327957c4b9ade1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.279086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd0238d8b217440e84327957c4b9ade1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.280353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd0238d8b217440e84327957c4b9ade1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.290933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd0238d8b217440e84327957c4b9ade1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.323555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1284/2544] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:43 2022] DELETE /identity/v3/users/cd0238d8b217440e84327957c4b9ade1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:43.328367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.328780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.329067 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.358534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:43.359735 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:43.428919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:43.456635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1261/2545] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:43.473892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:43.475590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:43.499816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:43.501595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:43.605005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1e348-ceb5-4618-99d3-3118c7933abe admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:43.618708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd1cc-77ba-46dd-9420-baf3240d8af4 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:43.621099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:43.621769 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.622108 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.622323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.625160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc920003220f417ea5bb59cb80e9d286', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba07646f5d243219b808a65fa0d4c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:43.625844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.626139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.626420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.627183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:43.627499 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:43.629344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:43.635911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:43.637786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:43.649314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1262/2546] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:38:43.649314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:43.649314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:43.654470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1285/2547] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:43.656090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:43.666855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853aaee-9d6b-49f5-b49e-b93d87d54f8a tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:43.669771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e005bcf-2fb2-4200-ad2d-fce5c06e8c1b service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:43.673294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:43.673958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.674468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.674602 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] PATH_INFO: `/v3/projects/3645666a6f8242808b7a5ddfe48d42a8` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.676040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3645666a6f8242808b7a5ddfe48d42a8)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:43.678527 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:43.679190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc920003220f417ea5bb59cb80e9d286', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba07646f5d243219b808a65fa0d4c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:43.680491 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.680979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.681775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.682726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:43.686129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:43.729573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1286/2548] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:43 2022] POST /identity/v3/projects => generated 354 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:43.734469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:43.776621 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3645666a6f8242808b7a5ddfe48d42a8', 'request_id': 'req-d115d77e-25a4-4385-97e1-48521d64e0a7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.786339 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3645666a6f8242808b7a5ddfe48d42a8', 'request_id': 'req-d115d77e-25a4-4385-97e1-48521d64e0a7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.799474 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3645666a6f8242808b7a5ddfe48d42a8', 'request_id': 'req-d115d77e-25a4-4385-97e1-48521d64e0a7'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.824360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] The token cache is being invalidate because project 3645666a6f8242808b7a5ddfe48d42a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:43.824664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.825392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d115d77e-25a4-4385-97e1-48521d64e0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:43.833305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1263/2549] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:43 2022] DELETE /identity/v3/projects/3645666a6f8242808b7a5ddfe48d42a8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:43.835080 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e115c2-e84e-4a16-9c9b-f6874f061369 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.835372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e115c2-e84e-4a16-9c9b-f6874f061369 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.835654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e115c2-e84e-4a16-9c9b-f6874f061369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.866366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-33e115c2-e84e-4a16-9c9b-f6874f061369 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:43.867770 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-33e115c2-e84e-4a16-9c9b-f6874f061369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:43.896047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02878208-e1b3-4466-9078-31e110dcf791 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:43.983876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1264/2550] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:43.993485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:43.993840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:43.994056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:43.996821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:44.010616 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc920003220f417ea5bb59cb80e9d286', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba07646f5d243219b808a65fa0d4c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.011524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.011996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.016817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.017623 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:44.017946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.019744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.025393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:44.026773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.041867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1287/2551] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:38:44.043151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1265/2552] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:44.056016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.092478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a59f8f-192e-4a12-ad5e-d70639dca6e5 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.094171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.095594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.098321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.099189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.099643 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.100114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.101004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:44.101346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.103007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.110392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:44.117006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281edc7d-f4fa-4c85-a6c9-b78b78fd3189 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.117658 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1288/2553] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:44.121756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.125211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc920003220f417ea5bb59cb80e9d286', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dba07646f5d243219b808a65fa0d4c4f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.126216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.126884 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.127274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] PATH_INFO: `/v3/projects/584e111bb27a41f99f939d2dadedd203` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.128746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorizing `identity:delete_project(project_id=584e111bb27a41f99f939d2dadedd203)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.131028 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.156473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c117402-4bb9-4fea-97e0-da009ed3d852 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.166984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.170387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.171116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.171547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.172448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.178198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.189691 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584e111bb27a41f99f939d2dadedd203', 'request_id': 'req-6bf44c63-09b3-4119-b810-8d3d9cf54359'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.196305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584e111bb27a41f99f939d2dadedd203', 'request_id': 'req-6bf44c63-09b3-4119-b810-8d3d9cf54359'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.206443 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584e111bb27a41f99f939d2dadedd203', 'request_id': 'req-6bf44c63-09b3-4119-b810-8d3d9cf54359'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.223614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] The token cache is being invalidate because project 584e111bb27a41f99f939d2dadedd203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:44.223907 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.224581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.232644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1266/2554] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:44 2022] DELETE /identity/v3/projects/584e111bb27a41f99f939d2dadedd203 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:44.232644 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.233848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.250528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1289/2555] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:44.255418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.258291 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.327892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf44c63-09b3-4119-b810-8d3d9cf54359 tempest-VolumeQuotaClassesTest-289719834 tempest-VolumeQuotaClassesTest-289719834-project-admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.367934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81e7f9-2eea-4dda-bb79-2de74a3a232c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.372859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.374249 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.374593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.374933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.375621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.377681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.411463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1290/2556] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:44.414928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.415978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.416329 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.416615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.416955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] PATH_INFO: `/v3/projects/e23b799cef7e44ffa8fd6278eec6396c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.418642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23b799cef7e44ffa8fd6278eec6396c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.421286 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.433756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127eebb-9fa6-497e-a104-8216e92bdbab admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.444618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.445333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.446009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.446009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.446915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.448808 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.462802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1291/2557] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:44.468603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.475827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:44.488900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61c1d0-f6c6-4228-8db3-96f07eb40924 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.493792 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23b799cef7e44ffa8fd6278eec6396c', 'request_id': 'req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.501168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23b799cef7e44ffa8fd6278eec6396c', 'request_id': 'req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.511144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23b799cef7e44ffa8fd6278eec6396c', 'request_id': 'req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.524436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] The token cache is being invalidate because project e23b799cef7e44ffa8fd6278eec6396c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:44.524832 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.525716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.528024 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1267/2558] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:44 2022] DELETE /identity/v3/projects/e23b799cef7e44ffa8fd6278eec6396c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:44.530290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.532015 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.555699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c6b79-769a-4f0a-9f18-8e8621151768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.556462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6b79-769a-4f0a-9f18-8e8621151768 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.556746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6b79-769a-4f0a-9f18-8e8621151768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.557018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6b79-769a-4f0a-9f18-8e8621151768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.557703 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c6b79-769a-4f0a-9f18-8e8621151768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.559417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c6b79-769a-4f0a-9f18-8e8621151768 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.573446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1292/2559] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:44.576725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.577021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.577347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.599702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:44.600937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.610151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1293/2560] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:44.610922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.612265 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.617744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e74f0-cf72-41d4-9cbc-ab43b8d8c924 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.624472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.625235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.625571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.625913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] PATH_INFO: `/v3/users/7cb96c148d3c4ec58b21eefd647b512c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.626020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc05db5-7f81-4396-b255-85055f2465a6 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.627198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb96c148d3c4ec58b21eefd647b512c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.629199 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.637218 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032bee7-bc16-4fb5-a6c7-521bb67b141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.637916 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032bee7-bc16-4fb5-a6c7-521bb67b141d admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.638197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032bee7-bc16-4fb5-a6c7-521bb67b141d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.638470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032bee7-bc16-4fb5-a6c7-521bb67b141d admin admin] PATH_INFO: `/v3/projects/2f6e2476f0ff47a5bea63978819c530f/users/8c00c8bff49f46d4b69b5d58b0d50652/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.650787 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:44.654086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032bee7-bc16-4fb5-a6c7-521bb67b141d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6e2476f0ff47a5bea63978819c530f, user_id=8c00c8bff49f46d4b69b5d58b0d50652, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.657859 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032bee7-bc16-4fb5-a6c7-521bb67b141d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.674138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1294/2561] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:44 2022] PUT /identity/v3/projects/2f6e2476f0ff47a5bea63978819c530f/users/8c00c8bff49f46d4b69b5d58b0d50652/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:44.675471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed667ba-b633-4ed1-8864-6eb560005bbb None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.675829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed667ba-b633-4ed1-8864-6eb560005bbb None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.676204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed667ba-b633-4ed1-8864-6eb560005bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.695594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb96c148d3c4ec58b21eefd647b512c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.696944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6ed667ba-b633-4ed1-8864-6eb560005bbb None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:44.698433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6ed667ba-b633-4ed1-8864-6eb560005bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.705063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb96c148d3c4ec58b21eefd647b512c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.716797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb96c148d3c4ec58b21eefd647b512c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.717740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb96c148d3c4ec58b21eefd647b512c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.718757 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-5380ee2d-d2ae-483b-b470-8aacf9eb4a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb96c148d3c4ec58b21eefd647b512c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:44.756927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1268/2562] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:44 2022] DELETE /identity/v3/users/7cb96c148d3c4ec58b21eefd647b512c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:44.762233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a130bc-0cfd-468b-a96d-b67d8071916c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.762634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a130bc-0cfd-468b-a96d-b67d8071916c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.763127 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a130bc-0cfd-468b-a96d-b67d8071916c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.795938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1295/2563] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:44.797236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.797560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.797886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.804981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e1a130bc-0cfd-468b-a96d-b67d8071916c None None] MFA Rules not processed for user `8c00c8bff49f46d4b69b5d58b0d50652`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:44.806265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e1a130bc-0cfd-468b-a96d-b67d8071916c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.819871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:44.821129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.841937 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:44.880548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1269/2564] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:44.881361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a130bc-0cfd-468b-a96d-b67d8071916c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.887442 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1296/2565] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:44.888520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.890129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.893749 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a130bc-0cfd-468b-a96d-b67d8071916c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.899007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.899985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.900330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.900668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.901420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:44.901791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.902826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff3003-afd9-45b6-b83a-1818199f8ccd None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:44.903521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.908454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:44.909056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:44.909507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:44.909665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:44.910320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:44.912272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:44.921165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1270/2566] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:44.921165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.921165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:44.946111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1297/2567] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:44.947349 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:44.986542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42dcb-47e8-4fa9-8ef5-51104f4ec132 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.016897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18888222-4fa2-45a7-ba77-6da5cffc720d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.021970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.022664 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.023047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.023420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.024156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.026248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.042441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.043751 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.044033 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:45.044194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.044692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.057526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.058937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1298/2568] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:44 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:45.059385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.059745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.070139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1271/2569] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:45.071264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.080942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31792945-c952-4360-9b23-af4f9b838888 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.159650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.160354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.160689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.161026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.161912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396d544-8c22-4513-80cc-af5a40acd388 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.162114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.164325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.167740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.168416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.168696 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.169106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.169775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.173135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.190013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1272/2570] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:45 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:45.194740 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.205695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061f20c-af9c-4146-9895-5c7fa1b07c4b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.210200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.210876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.211226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.211586 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.212296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.216051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1299/2571] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:45 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:45.216202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.217726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.218885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:45.230715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13879b-5060-406e-82bf-12ff73c87673 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.238916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1273/2572] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:45.238916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.241922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.242652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.243521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.243521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.244162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.246064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.254229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d6ea-0141-496e-ad3c-4cd4ea105cd9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.262386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1300/2573] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:45.264854 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.265231 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.266318 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.266870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.269784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.271191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.274461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.277536 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309bc21-0fde-40cd-9f11-cbeede5a69eb admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.283550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.284403 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.284756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.285201 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] PATH_INFO: `/v3/projects/2587cbbc657e4466988012f884a7d8d0/users/312caaae26dd404b9533e621ce441983/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.301009 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2587cbbc657e4466988012f884a7d8d0, user_id=312caaae26dd404b9533e621ce441983, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.301658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1274/2574] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:45.303091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.304684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.322896 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e904f-5ec4-49f1-b1bc-4746a51022ff admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.327126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1301/2575] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:45 2022] PUT /identity/v3/projects/2587cbbc657e4466988012f884a7d8d0/users/312caaae26dd404b9533e621ce441983/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:45.331155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.402623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.403299 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.403754 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.404040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] PATH_INFO: `/v3/projects/25007538371040b09fa5dd940ac7a6c3/users/af5cbf7c8de043aa802078cb0e0a4afa/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.430563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9179429-e4a3-4726-b119-52c9a0d4d2ac admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.431432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25007538371040b09fa5dd940ac7a6c3, user_id=af5cbf7c8de043aa802078cb0e0a4afa, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.434136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.437101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.437973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.438417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.438869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.439801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.442750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.457313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1302/2576] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:45.461368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.465457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1275/2577] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:45 2022] PUT /identity/v3/projects/25007538371040b09fa5dd940ac7a6c3/users/af5cbf7c8de043aa802078cb0e0a4afa/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:45.465457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.470878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:45.490548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e49bc1-41a9-4c7a-84e2-b84f2be11a99 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.532928 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5584-e30d-4005-ae29-2ab9fa295e89 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.535681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:45.537818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8edca81-6dca-4451-9fe9-5d1e631644a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.538579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edca81-6dca-4451-9fe9-5d1e631644a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.538887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edca81-6dca-4451-9fe9-5d1e631644a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.539215 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edca81-6dca-4451-9fe9-5d1e631644a8 admin admin] PATH_INFO: `/v3/projects/2587cbbc657e4466988012f884a7d8d0/users/312caaae26dd404b9533e621ce441983/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.541288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8edca81-6dca-4451-9fe9-5d1e631644a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2587cbbc657e4466988012f884a7d8d0, user_id=312caaae26dd404b9533e621ce441983, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.544124 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8edca81-6dca-4451-9fe9-5d1e631644a8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.549357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.550013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.550285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.550562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] PATH_INFO: `/v3/projects/20a373b8272441549f04aad0f0677667` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.551705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a373b8272441549f04aad0f0677667)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.553701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.554866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1276/2578] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:45 2022] PUT /identity/v3/projects/2587cbbc657e4466988012f884a7d8d0/users/312caaae26dd404b9533e621ce441983/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:45.555861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.556147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.556427 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.594518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] MFA Rules not processed for user `af5cbf7c8de043aa802078cb0e0a4afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:45.596020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:45.625799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a373b8272441549f04aad0f0677667', 'request_id': 'req-48e515db-8bfe-452b-a102-34e3ebcd6389'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.632081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a373b8272441549f04aad0f0677667', 'request_id': 'req-48e515db-8bfe-452b-a102-34e3ebcd6389'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.640919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a373b8272441549f04aad0f0677667', 'request_id': 'req-48e515db-8bfe-452b-a102-34e3ebcd6389'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.653290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] The token cache is being invalidate because project 20a373b8272441549f04aad0f0677667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:45.653508 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.654168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-48e515db-8bfe-452b-a102-34e3ebcd6389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.657441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1303/2579] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:45 2022] DELETE /identity/v3/projects/20a373b8272441549f04aad0f0677667 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:45.657441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.657441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.657728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.664723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:45.680447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1277/2580] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:45.681666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.683191 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:45.687360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] MFA Rules not processed for user `312caaae26dd404b9533e621ce441983`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:45.688606 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:45.761997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04b1cd-71c1-47d3-baf3-cf5798b7e39e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.765355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1304/2581] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:45.770317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.772193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:45.830601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.831305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.831588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.831859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] PATH_INFO: `/v3/users/fc920003220f417ea5bb59cb80e9d286` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.833039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc920003220f417ea5bb59cb80e9d286)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.833812 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:45.834989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.852675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478416-4a4a-447b-adee-2e095550e178 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:45.859737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:45.860260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:45.860583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:45.860791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:45.892927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc920003220f417ea5bb59cb80e9d286'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.901105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc920003220f417ea5bb59cb80e9d286'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.911549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc920003220f417ea5bb59cb80e9d286'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.912596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc920003220f417ea5bb59cb80e9d286'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.913487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc920003220f417ea5bb59cb80e9d286'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:45.924914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:45.926677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:45.933384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1278/2582] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:45 2022] DELETE /identity/v3/users/fc920003220f417ea5bb59cb80e9d286 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:45.934134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:45.935572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:45.941010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1305/2583] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:45.959893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:46.007260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fae69f-59c1-4c59-9663-9fba589700d5 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.011239 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.011502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.011781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.014983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.014983 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.103425 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.105020 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.118072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1279/2584] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:46.212256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.212256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.221666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.223360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.230925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d18822-dd75-4646-a0a7-ce4072e0f775 service nova] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.303177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.304062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.304492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.304911 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] PATH_INFO: `/v3/users/af5cbf7c8de043aa802078cb0e0a4afa` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.306745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5cbf7c8de043aa802078cb0e0a4afa)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.309190 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.317112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59badd-3e71-4c5b-9eb8-c51329463d72 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.321829 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.322458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.322742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.323021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] PATH_INFO: `/v3/users/8c00c8bff49f46d4b69b5d58b0d50652` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.324193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c00c8bff49f46d4b69b5d58b0d50652)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.326272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.370422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5cbf7c8de043aa802078cb0e0a4afa'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.382947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5cbf7c8de043aa802078cb0e0a4afa'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.383144 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c00c8bff49f46d4b69b5d58b0d50652'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.383246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5cbf7c8de043aa802078cb0e0a4afa'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.383246 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5cbf7c8de043aa802078cb0e0a4afa'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.389661 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c00c8bff49f46d4b69b5d58b0d50652'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.397200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5cbf7c8de043aa802078cb0e0a4afa'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.399903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c00c8bff49f46d4b69b5d58b0d50652'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.400698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c00c8bff49f46d4b69b5d58b0d50652'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.401413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-36cf8ef9-71c5-4ce1-92e3-86e924a35a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c00c8bff49f46d4b69b5d58b0d50652'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.414408 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:46.423104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1306/2585] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:46 2022] DELETE /identity/v3/users/af5cbf7c8de043aa802078cb0e0a4afa => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:46.423965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.427225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.428003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1280/2586] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:46 2022] DELETE /identity/v3/users/8c00c8bff49f46d4b69b5d58b0d50652 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:46.429090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2f5b-3b87-441f-8bdd-ba0273ae5f5a None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.429429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2f5b-3b87-441f-8bdd-ba0273ae5f5a None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.429713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2f5b-3b87-441f-8bdd-ba0273ae5f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.450974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:46.454707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-168d2f5b-3b87-441f-8bdd-ba0273ae5f5a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:46.455946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-168d2f5b-3b87-441f-8bdd-ba0273ae5f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.500935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a333a80-9156-4688-aa19-217e095c6875 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.522995 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1281/2587] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:46.524519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.524916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.525356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.548184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:46.549793 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.559194 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1282/2588] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:46.559667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.560021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.560432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.561342 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.561695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] PATH_INFO: `/v3/projects/dba07646f5d243219b808a65fa0d4c4f` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.562370 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.563496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba07646f5d243219b808a65fa0d4c4f)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.565759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.630014 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba07646f5d243219b808a65fa0d4c4f', 'request_id': 'req-5cb97ed5-c224-4f78-9870-bc3291296d39'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.636242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba07646f5d243219b808a65fa0d4c4f', 'request_id': 'req-5cb97ed5-c224-4f78-9870-bc3291296d39'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.643164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a592cc-143d-4e08-b2a0-d6ab8538f1fe None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.647776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba07646f5d243219b808a65fa0d4c4f', 'request_id': 'req-5cb97ed5-c224-4f78-9870-bc3291296d39'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.659848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] The token cache is being invalidate because project dba07646f5d243219b808a65fa0d4c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:46.660136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.660836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:46.662587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1307/2589] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:46 2022] DELETE /identity/v3/projects/dba07646f5d243219b808a65fa0d4c4f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:46.663173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.664567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.704822 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb786dc-6be2-4925-8d10-ffbb3d1e7d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.705583 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb786dc-6be2-4925-8d10-ffbb3d1e7d31 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.705887 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb786dc-6be2-4925-8d10-ffbb3d1e7d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.706158 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb786dc-6be2-4925-8d10-ffbb3d1e7d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.707926 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-dbb786dc-6be2-4925-8d10-ffbb3d1e7d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.723863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97ed5-c224-4f78-9870-bc3291296d39 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.728238 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148ce75-3a1e-47c9-acaf-8406aa973f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.728891 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148ce75-3a1e-47c9-acaf-8406aa973f85 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.729168 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148ce75-3a1e-47c9-acaf-8406aa973f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.729477 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148ce75-3a1e-47c9-acaf-8406aa973f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.731272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-f148ce75-3a1e-47c9-acaf-8406aa973f85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.741126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:46.760616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:46.773613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb786dc-6be2-4925-8d10-ffbb3d1e7d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.775051 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb786dc-6be2-4925-8d10-ffbb3d1e7d31 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.784546 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1283/2590] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:46.785784 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6531ae-38b7-45c4-a1c6-7d91ba09396d None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.786170 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6531ae-38b7-45c4-a1c6-7d91ba09396d None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.786450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6531ae-38b7-45c4-a1c6-7d91ba09396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.798459 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148ce75-3a1e-47c9-acaf-8406aa973f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.800409 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148ce75-3a1e-47c9-acaf-8406aa973f85 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.806338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-6b6531ae-38b7-45c4-a1c6-7d91ba09396d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:46.807613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-6b6531ae-38b7-45c4-a1c6-7d91ba09396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.811649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1308/2591] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:46.816486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1284/2592] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:46.824083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24dd23-3bef-4ac3-ba08-aa91a0588fc1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.824370 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24dd23-3bef-4ac3-ba08-aa91a0588fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.824644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24dd23-3bef-4ac3-ba08-aa91a0588fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.844541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fb24dd23-3bef-4ac3-ba08-aa91a0588fc1 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:46.846057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fb24dd23-3bef-4ac3-ba08-aa91a0588fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:46.854674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1309/2593] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:46.859605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6531ae-38b7-45c4-a1c6-7d91ba09396d None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.870073 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6531ae-38b7-45c4-a1c6-7d91ba09396d None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.876207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.876868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.877148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.877453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.878101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:46.878400 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.880729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.892932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1285/2594] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:46.897427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24dd23-3bef-4ac3-ba08-aa91a0588fc1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.907093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24dd23-3bef-4ac3-ba08-aa91a0588fc1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:46.911755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:46.912357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:46.912667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:46.912978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:46.913624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:46.915645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:46.945726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1310/2595] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:46 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:46.949966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:46.960982 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab61f6-e358-4e40-8419-30a40a1be9f9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.021715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.022319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.022596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.022990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.023623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.026362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.051468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1286/2596] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:46 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:47.056150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.069601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b64fe7-44db-49ad-b358-71d11ab05416 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.074673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.075430 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.075547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.075799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.076482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.078112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.088550 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1311/2597] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:47.093155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.103739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3fbd1-a693-4c86-b345-fca73f98bf68 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.109322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482e842-0e15-4abb-a932-fb9e0fa3f14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.110125 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e842-0e15-4abb-a932-fb9e0fa3f14b admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.110530 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e842-0e15-4abb-a932-fb9e0fa3f14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.110808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482e842-0e15-4abb-a932-fb9e0fa3f14b admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.111447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482e842-0e15-4abb-a932-fb9e0fa3f14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.113027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482e842-0e15-4abb-a932-fb9e0fa3f14b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.124256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1287/2598] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:47.128484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.139413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0082c70-0e5d-4793-8123-2feb2f0d37f0 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.143976 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.144624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.144908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.145186 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] PATH_INFO: `/v3/projects/2695f3781a9b496f87a3ebef7d2f2010/users/fe8cd4bd3a194e72b4f77b86343635c6/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.161548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2695f3781a9b496f87a3ebef7d2f2010, user_id=fe8cd4bd3a194e72b4f77b86343635c6, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.164258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.178045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1312/2599] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:47 2022] PUT /identity/v3/projects/2695f3781a9b496f87a3ebef7d2f2010/users/fe8cd4bd3a194e72b4f77b86343635c6/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:47.188413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.188698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.188968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.219775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] MFA Rules not processed for user `fe8cd4bd3a194e72b4f77b86343635c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:47.221293 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.257713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.259811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.312516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1288/2600] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:47.317312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.318864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.335884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d4e62-ae43-45ea-b890-afef050b9f33 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.340762 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.341531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.341818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.342625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] PATH_INFO: `/v3/projects/25007538371040b09fa5dd940ac7a6c3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.343804 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25007538371040b09fa5dd940ac7a6c3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.346037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.361140 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:47.388212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10d9d-71b6-48db-ad7d-932eb018cdfc None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.390234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:47.394242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25007538371040b09fa5dd940ac7a6c3', 'request_id': 'req-6e424191-d0f6-43fa-a642-b7fae379a084'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.401830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25007538371040b09fa5dd940ac7a6c3', 'request_id': 'req-6e424191-d0f6-43fa-a642-b7fae379a084'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.410905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25007538371040b09fa5dd940ac7a6c3', 'request_id': 'req-6e424191-d0f6-43fa-a642-b7fae379a084'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.424138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] The token cache is being invalidate because project 25007538371040b09fa5dd940ac7a6c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:47.424426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.425079 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.426608 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1313/2601] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:47 2022] DELETE /identity/v3/projects/25007538371040b09fa5dd940ac7a6c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:47.427377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.428776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.448393 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.449533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.449886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.450232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] PATH_INFO: `/v3/projects/2f6e2476f0ff47a5bea63978819c530f` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.452305 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6e2476f0ff47a5bea63978819c530f)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.454367 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.493974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e424191-d0f6-43fa-a642-b7fae379a084 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.503356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6e2476f0ff47a5bea63978819c530f', 'request_id': 'req-166a8633-9dc6-4d0f-b097-ec7aa2e89713'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.508858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6e2476f0ff47a5bea63978819c530f', 'request_id': 'req-166a8633-9dc6-4d0f-b097-ec7aa2e89713'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.516724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6e2476f0ff47a5bea63978819c530f', 'request_id': 'req-166a8633-9dc6-4d0f-b097-ec7aa2e89713'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.527894 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] The token cache is being invalidate because project 2f6e2476f0ff47a5bea63978819c530f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:47.528186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.528831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-166a8633-9dc6-4d0f-b097-ec7aa2e89713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.531040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1289/2602] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:47 2022] DELETE /identity/v3/projects/2f6e2476f0ff47a5bea63978819c530f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:47.532219 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.532501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.532769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.550126 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87596c15-fcd4-4d2d-92f2-5aa6c9147ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.550740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87596c15-fcd4-4d2d-92f2-5aa6c9147ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.551019 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87596c15-fcd4-4d2d-92f2-5aa6c9147ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.551292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87596c15-fcd4-4d2d-92f2-5aa6c9147ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.551818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:47.551922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87596c15-fcd4-4d2d-92f2-5aa6c9147ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.553006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.553850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87596c15-fcd4-4d2d-92f2-5aa6c9147ba2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.560706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1290/2603] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:47.561532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.562717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.574625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a472d90-9767-40a2-8c01-1f0c634a2071 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.580065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.580805 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.581172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.581570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] PATH_INFO: `/v3/users/312caaae26dd404b9533e621ce441983` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.582214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1314/2604] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:47 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:47.582968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] RBAC: Authorizing `identity:delete_user(user_id=312caaae26dd404b9533e621ce441983)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.583502 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.583791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.584066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.585521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.606700 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:47.607922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.635775 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312caaae26dd404b9533e621ce441983'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.644883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312caaae26dd404b9533e621ce441983'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.652592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312caaae26dd404b9533e621ce441983'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.653524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312caaae26dd404b9533e621ce441983'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.654146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312caaae26dd404b9533e621ce441983'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:47.655919 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:47.671634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1315/2605] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:47.672522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.673926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:47.676270 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1291/2606] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:47 2022] DELETE /identity/v3/users/312caaae26dd404b9533e621ce441983 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:47.677168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.682006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:47.686234 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eaee67-9ae3-4e43-b8ed-922e41cd2009 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.759657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dc542-598c-403f-8a72-186a808a280c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.768301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.769112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.769915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.770264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.771066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:47.771386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.773911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.788200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.789143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.789143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.789406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.790006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.792078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.793726 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1292/2607] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:47.797948 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.810671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e267061-fc3f-4af3-8cbe-96c92d48b2e6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.817465 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.818223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.818572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.818915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.819649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.821815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.828310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1316/2608] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:47 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:47.832412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.842598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c51b0-3dbb-477e-839c-0b0ce845158e admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.848085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.848702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.848986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.849297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.849950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.851575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.868337 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1317/2609] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:47.869710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.873790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1293/2610] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:47 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:47.877850 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.883311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ee76-5243-4fb7-bdfc-6d44dfeb52e6 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.952603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.953382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.953729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.954073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] PATH_INFO: `/v3/projects/4ff7468ed70c4d7f94053aa3167375e0/users/2845504fb48548a1b65a72242822b4ab/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.958518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b88b-615c-4d78-a816-1086834bd0f3 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:47.963653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:47.964574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:47.964574 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:47.964876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:47.965517 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.967533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.970150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff7468ed70c4d7f94053aa3167375e0, user_id=2845504fb48548a1b65a72242822b4ab, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:47.973160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:47.987487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1318/2611] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:47 2022] PUT /identity/v3/projects/4ff7468ed70c4d7f94053aa3167375e0/users/2845504fb48548a1b65a72242822b4ab/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:47.991060 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:47.993312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1294/2612] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:47 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:47.996879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.013314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab178e-ac33-49be-a068-185f49523e71 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.066264 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9515eb-9146-400b-a2b3-aa252bb03c46 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.067720 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:48.071110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.071722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.072003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.072327 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.073004 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.074838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.085487 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1319/2613] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:48.086369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.088023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.094195 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.094807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.095089 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.095363 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.096013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:48.096327 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.098014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.102845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327e8f9-b858-42c9-9ae3-c8d93550332a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.108331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.109023 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1295/2614] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:48.109204 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.109684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.109886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.110072 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] PATH_INFO: `/v3/projects/2587cbbc657e4466988012f884a7d8d0` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.111485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2587cbbc657e4466988012f884a7d8d0)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.113923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.121915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2fd90-7a6d-4799-9575-f76eda00460c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.128018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.128551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.129126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.129126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] PATH_INFO: `/v3/projects/4ff7468ed70c4d7f94053aa3167375e0/users/2845504fb48548a1b65a72242822b4ab/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.131082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff7468ed70c4d7f94053aa3167375e0, user_id=2845504fb48548a1b65a72242822b4ab, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.133853 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.146398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1296/2615] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:48 2022] PUT /identity/v3/projects/4ff7468ed70c4d7f94053aa3167375e0/users/2845504fb48548a1b65a72242822b4ab/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:48.147382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.166283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2587cbbc657e4466988012f884a7d8d0', 'request_id': 'req-00f2b4ec-72fd-4bf1-a887-1d06e0207704'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:48.171851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2587cbbc657e4466988012f884a7d8d0', 'request_id': 'req-00f2b4ec-72fd-4bf1-a887-1d06e0207704'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:48.180290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2587cbbc657e4466988012f884a7d8d0', 'request_id': 'req-00f2b4ec-72fd-4bf1-a887-1d06e0207704'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:48.194150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] The token cache is being invalidate because project 2587cbbc657e4466988012f884a7d8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:48.194445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:48.195217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-00f2b4ec-72fd-4bf1-a887-1d06e0207704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:48.196687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1320/2616] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:48 2022] DELETE /identity/v3/projects/2587cbbc657e4466988012f884a7d8d0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:48.197974 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.198286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.198565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.215732 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853fd26-4e64-443c-85e6-5589543da7be admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.227269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] MFA Rules not processed for user `2845504fb48548a1b65a72242822b4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:48.228509 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.273483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f596e-fc3b-4ac7-9f8d-c36cc2b6e8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.274090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f596e-fc3b-4ac7-9f8d-c36cc2b6e8eb admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.274378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f596e-fc3b-4ac7-9f8d-c36cc2b6e8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.274656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f596e-fc3b-4ac7-9f8d-c36cc2b6e8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.275301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f596e-fc3b-4ac7-9f8d-c36cc2b6e8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.276923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f596e-fc3b-4ac7-9f8d-c36cc2b6e8eb admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.286668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1297/2617] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:48.286668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.286668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.286668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.302721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1321/2618] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:48.309048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:48.309967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.310476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.312384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.319508 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1298/2619] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:48.320316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.327396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899e10-561b-4d67-9a84-1b75492a3e40 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.328950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.339539 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe52fb-7c18-406f-9328-936f0ce2d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.340505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe52fb-7c18-406f-9328-936f0ce2d83e admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.340783 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe52fb-7c18-406f-9328-936f0ce2d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.341049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe52fb-7c18-406f-9328-936f0ce2d83e admin admin] PATH_INFO: `/v3/projects/ffaa07497b7f49b8bc60bfdec38abffa/users/6a931badb30645ceb34ea383040523fd/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.359549 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe52fb-7c18-406f-9328-936f0ce2d83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffaa07497b7f49b8bc60bfdec38abffa, user_id=6a931badb30645ceb34ea383040523fd, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.362325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe52fb-7c18-406f-9328-936f0ce2d83e admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.377374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1322/2620] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:48 2022] PUT /identity/v3/projects/ffaa07497b7f49b8bc60bfdec38abffa/users/6a931badb30645ceb34ea383040523fd/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:48.387617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e479f4e-6f17-4a86-8a43-6e82bd71d3ba None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.387912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e479f4e-6f17-4a86-8a43-6e82bd71d3ba None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.388164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e479f4e-6f17-4a86-8a43-6e82bd71d3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.397838 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0cac4-dc21-468f-bd3a-4a091b11b604 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.407641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7e479f4e-6f17-4a86-8a43-6e82bd71d3ba None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:48.408909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7e479f4e-6f17-4a86-8a43-6e82bd71d3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.411836 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:48.458838 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:48.462596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.463212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.463514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.463947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.478332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1323/2621] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:48.481956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51af5e2-24cc-477f-8f6d-676ebbd9952d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.481956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51af5e2-24cc-477f-8f6d-676ebbd9952d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.481956 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51af5e2-24cc-477f-8f6d-676ebbd9952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.509466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a51af5e2-24cc-477f-8f6d-676ebbd9952d None None] MFA Rules not processed for user `6a931badb30645ceb34ea383040523fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:48.510684 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a51af5e2-24cc-477f-8f6d-676ebbd9952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.525653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.527098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.536592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1299/2622] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:48.537405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.547392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00f9f-c864-4485-ba0f-50f20b735102 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.552090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.552692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.552960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.553231 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.553901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:48.554205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.555778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.571232 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1300/2623] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:48.585600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1324/2624] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:48.585716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.585813 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51af5e2-24cc-477f-8f6d-676ebbd9952d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.595324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f232-4b85-4644-b49c-b29ab51e3293 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.596354 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51af5e2-24cc-477f-8f6d-676ebbd9952d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.600578 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.601187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.601504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.601779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.601961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.602405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.602527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.602897 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.603091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.603822 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.604803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.607278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.644093 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1301/2625] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:48.645325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1325/2626] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:48.645453 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.646556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.721645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e32ead-a72f-4eb2-bad5-d6252932615c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.726348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.726973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.727269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.727593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.728186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.731309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.736748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b211-e8fe-44d6-b3dc-65f8c9247a44 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.742471 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.743239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.743569 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.743826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.744506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.746961 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.755273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1302/2627] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:48.759404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.771696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435966d2-2b80-463a-aeb8-15d0d57d4477 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.774839 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1326/2628] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:48.775938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.781324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.782068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.782208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.782485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.783132 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.784745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.790270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e7f9a-bc82-4fe4-b5cf-6b7a4c7d9456 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.794718 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.795914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.796193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.796466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.797112 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.798138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1303/2629] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:48.799013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.801672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.810097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1327/2630] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:48.811789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a4d2e-bcb3-4d75-8d79-3535ea3ed603 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.814423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.816759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.817511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.817827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.818325 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.819164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.821418 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.829731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:48.829886 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90498-4086-4a11-86af-ee10ff89b3b3 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.835139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.835801 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.836078 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.836350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] PATH_INFO: `/v3/projects/a6045c3f4a0f430196258e295cf3efce/users/0cac601e019745dc8e774c7dc689d57a/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.836484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1304/2631] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:48.840833 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.853908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891a4fc-f1f6-4548-a38d-7a3f37a6a888 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:48.855631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6045c3f4a0f430196258e295cf3efce, user_id=0cac601e019745dc8e774c7dc689d57a, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.859144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.859263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9477b-daa0-4afc-a8d7-cff151249ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:48.863779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9477b-daa0-4afc-a8d7-cff151249ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.863779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9477b-daa0-4afc-a8d7-cff151249ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.863779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9477b-daa0-4afc-a8d7-cff151249ad4 admin admin] PATH_INFO: `/v3/projects/03da2be4f7334fa2a74c281b64e1be21/users/cb5912bdede54cb8a0a0c53e91db8cb2/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.876389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1328/2632] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:48 2022] PUT /identity/v3/projects/a6045c3f4a0f430196258e295cf3efce/users/0cac601e019745dc8e774c7dc689d57a/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:48.877454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9477b-daa0-4afc-a8d7-cff151249ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03da2be4f7334fa2a74c281b64e1be21, user_id=cb5912bdede54cb8a0a0c53e91db8cb2, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:48.880146 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9477b-daa0-4afc-a8d7-cff151249ad4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:48.880263 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:48.892066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:48.896088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1305/2633] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:48 2022] PUT /identity/v3/projects/03da2be4f7334fa2a74c281b64e1be21/users/cb5912bdede54cb8a0a0c53e91db8cb2/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:48.906856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:48.907210 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:48.907570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:48.937966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] MFA Rules not processed for user `cb5912bdede54cb8a0a0c53e91db8cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:48.938997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:48.969888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950dc85-ed7f-424e-a703-56f474106a19 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.020844 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1306/2634] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:49.022006 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.023394 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.042486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.043230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.043868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.044361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.045199 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.047144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.061109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1329/2635] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:49.061109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.070770 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07193cb-12fe-4908-b9dd-1ee887f2b23f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.078579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.079269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.079545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.079938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.080630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.082673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.114314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55a8a3-500e-49d7-b6d7-3bbe20556451 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.118652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1330/2636] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:49 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:49.119681 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.125207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.126083 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.126432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.126777 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] PATH_INFO: `/v3/users/fe8cd4bd3a194e72b4f77b86343635c6` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.128615 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8cd4bd3a194e72b4f77b86343635c6)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.130619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.178605 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8cd4bd3a194e72b4f77b86343635c6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.185500 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8cd4bd3a194e72b4f77b86343635c6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.187205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9f1c9-6b1b-4283-a80d-de5f0db1f5cc admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.192791 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.193560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8cd4bd3a194e72b4f77b86343635c6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.193713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.193923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.194200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] PATH_INFO: `/v3/projects/a6045c3f4a0f430196258e295cf3efce/users/0cac601e019745dc8e774c7dc689d57a/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.194315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8cd4bd3a194e72b4f77b86343635c6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.194942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-fd004001-e8d2-4e2f-a740-421fb0af4ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8cd4bd3a194e72b4f77b86343635c6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.196148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6045c3f4a0f430196258e295cf3efce, user_id=0cac601e019745dc8e774c7dc689d57a, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.198832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.199724 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:49.211163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1331/2637] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:49 2022] PUT /identity/v3/projects/a6045c3f4a0f430196258e295cf3efce/users/0cac601e019745dc8e774c7dc689d57a/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:49.211958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.213641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.218396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1307/2638] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:49 2022] DELETE /identity/v3/users/fe8cd4bd3a194e72b4f77b86343635c6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:49.223437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac6711-b328-4a23-9cab-eec2bf598dc0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.223723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac6711-b328-4a23-9cab-eec2bf598dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.223999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac6711-b328-4a23-9cab-eec2bf598dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.244056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:49.252446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5cac6711-b328-4a23-9cab-eec2bf598dc0 None None] MFA Rules not processed for user `0cac601e019745dc8e774c7dc689d57a`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:49.253692 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5cac6711-b328-4a23-9cab-eec2bf598dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.284682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60db3-560b-483e-a4d8-4ba4f11cc878 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.324126 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1308/2639] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:49.327488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.327779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.328055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.342879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.343500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.343781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.344062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.344692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.346943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.348487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:49.349713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.357209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1309/2640] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:49.361595 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.363014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.372834 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1332/2641] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:49 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:49.374235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.377612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.425671 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9694-9a2d-4d6c-8fd7-2b2d5f7c1653 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.429466 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.430075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.430356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.430630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.437430 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e858e72-f74a-4493-88ee-7dd95e563b26 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.440444 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.441095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.441739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.441908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.442023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.442273 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.451356 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.452129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1310/2642] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:49.452944 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.453852 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.464164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1333/2643] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:49.464352 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c0816-3b17-4934-a67f-cb7964ce9347 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.469347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.470025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.470302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.470494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.471405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.472997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.486727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.486873 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1311/2644] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:49.486959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.487654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.497762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33365476-9f10-46f5-8ff8-718961242f7e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.500533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f442d-1660-447e-a8e1-86c11881e9f4 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.503733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.504082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.504450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.504729 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] PATH_INFO: `/v3/projects/e4bcbe058eb14823ad96fe547313cfee/users/ac31db49176f4bb8a733e5ba9d8654e9/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.505295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.506041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.506574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.506574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] PATH_INFO: `/v3/users/6a931badb30645ceb34ea383040523fd` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.508712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a931badb30645ceb34ea383040523fd)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.510695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.521973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4bcbe058eb14823ad96fe547313cfee, user_id=ac31db49176f4bb8a733e5ba9d8654e9, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.525401 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.540337 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1312/2645] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:49 2022] PUT /identity/v3/projects/e4bcbe058eb14823ad96fe547313cfee/users/ac31db49176f4bb8a733e5ba9d8654e9/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:49.544172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.577590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a931badb30645ceb34ea383040523fd'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.587522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a931badb30645ceb34ea383040523fd'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.588202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a931badb30645ceb34ea383040523fd'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.588781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a931badb30645ceb34ea383040523fd'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.590426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:49.596810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:49.600406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5367fcd6-ebc0-4601-9062-0e833fa736ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a931badb30645ceb34ea383040523fd'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:49.632658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e4c88-93cd-4e14-a52e-31d10d4e453d admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.638406 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.639759 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.640013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.640314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.641476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.643161 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.646746 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1334/2646] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:49 2022] DELETE /identity/v3/users/6a931badb30645ceb34ea383040523fd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:49.651085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.651373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.651645 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.654744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1313/2647] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:49.658477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.660179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.672384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:49.673709 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.675976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88690f-8af9-455e-834d-a145d77d8818 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.738369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1335/2648] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:49.740369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550cb2e-6240-4ec3-a040-7997a04e7007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.741175 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550cb2e-6240-4ec3-a040-7997a04e7007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.741486 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550cb2e-6240-4ec3-a040-7997a04e7007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.741899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550cb2e-6240-4ec3-a040-7997a04e7007 admin admin] PATH_INFO: `/v3/projects/e4bcbe058eb14823ad96fe547313cfee/users/ac31db49176f4bb8a733e5ba9d8654e9/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.743849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550cb2e-6240-4ec3-a040-7997a04e7007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4bcbe058eb14823ad96fe547313cfee, user_id=ac31db49176f4bb8a733e5ba9d8654e9, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.746881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550cb2e-6240-4ec3-a040-7997a04e7007 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.747693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.749884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.762848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1314/2649] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:49 2022] PUT /identity/v3/projects/e4bcbe058eb14823ad96fe547313cfee/users/ac31db49176f4bb8a733e5ba9d8654e9/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:49.774769 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.775050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.775445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.812885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] MFA Rules not processed for user `ac31db49176f4bb8a733e5ba9d8654e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:49.814366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.821281 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb91eb1-eac5-4f14-a6b5-b59a4c080291 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.871754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.872363 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.872638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.872906 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.888284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1315/2650] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:49.889911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.891814 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.919466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:49.936967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.938473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:49.942623 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:49.948485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1336/2651] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:49.949278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:49.950591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:49.961708 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f66e3-d8af-43bc-be29-4e5576592c17 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:49.966791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:49.967449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:49.967848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:49.968253 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] PATH_INFO: `/v3/projects/2695f3781a9b496f87a3ebef7d2f2010` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:49.969864 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2695f3781a9b496f87a3ebef7d2f2010)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:49.972182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.040608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2695f3781a9b496f87a3ebef7d2f2010', 'request_id': 'req-22f347c7-66d2-46a6-a62d-c86405af1f8b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.048316 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2695f3781a9b496f87a3ebef7d2f2010', 'request_id': 'req-22f347c7-66d2-46a6-a62d-c86405af1f8b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.057634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2695f3781a9b496f87a3ebef7d2f2010', 'request_id': 'req-22f347c7-66d2-46a6-a62d-c86405af1f8b'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.073730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] The token cache is being invalidate because project 2695f3781a9b496f87a3ebef7d2f2010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:50.074107 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.074794 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.077402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1316/2652] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:49 2022] DELETE /identity/v3/projects/2695f3781a9b496f87a3ebef7d2f2010 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:50.081772 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:50.083351 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.156904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebdaf3-bed7-489c-b802-725ae1df765d service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:50.180364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f347c7-66d2-46a6-a62d-c86405af1f8b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:50.186672 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:50.187319 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.187598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.187939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] PATH_INFO: `/v3/users/2845504fb48548a1b65a72242822b4ab` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.189292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2845504fb48548a1b65a72242822b4ab)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:50.192272 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.219138 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:50.219660 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.219937 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.220207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.222587 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.252254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2845504fb48548a1b65a72242822b4ab'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.264372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2845504fb48548a1b65a72242822b4ab'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.274881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2845504fb48548a1b65a72242822b4ab'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.276129 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2845504fb48548a1b65a72242822b4ab'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.277281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2845504fb48548a1b65a72242822b4ab'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.292200 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:50.297617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:50.299528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.304634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1317/2653] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:50 2022] DELETE /identity/v3/users/2845504fb48548a1b65a72242822b4ab => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:50.314733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1337/2654] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:50.349516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:50.349516 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.457004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cdf09-2244-45fb-a9f4-47b3ac487a6a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:50.462876 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:50.463522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.463798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.464070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] PATH_INFO: `/v3/projects/ffaa07497b7f49b8bc60bfdec38abffa` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.465731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffaa07497b7f49b8bc60bfdec38abffa)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:50.468209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.503367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:50.504820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.518851 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbaac6-2a3a-41a7-abd6-ffdb6bf31a4c service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:50.525423 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:50.526100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.526387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.526661 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] PATH_INFO: `/v3/users/cb5912bdede54cb8a0a0c53e91db8cb2` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.527858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5912bdede54cb8a0a0c53e91db8cb2)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:50.530441 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.539773 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffaa07497b7f49b8bc60bfdec38abffa', 'request_id': 'req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.546302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffaa07497b7f49b8bc60bfdec38abffa', 'request_id': 'req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.556809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffaa07497b7f49b8bc60bfdec38abffa', 'request_id': 'req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.571155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] The token cache is being invalidate because project ffaa07497b7f49b8bc60bfdec38abffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:50.571350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.571883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.574241 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1318/2655] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:50 2022] DELETE /identity/v3/projects/ffaa07497b7f49b8bc60bfdec38abffa => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:50.579638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:50.581019 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.585405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:50.589761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5912bdede54cb8a0a0c53e91db8cb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.602340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5912bdede54cb8a0a0c53e91db8cb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.603924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5912bdede54cb8a0a0c53e91db8cb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.604734 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5912bdede54cb8a0a0c53e91db8cb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.620989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5912bdede54cb8a0a0c53e91db8cb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.642397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1338/2656] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:50 2022] DELETE /identity/v3/users/cb5912bdede54cb8a0a0c53e91db8cb2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:50.643235 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:50.646030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.660216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0094-98c3-49b2-9f8d-e56d45f79aa6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:50.692956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:50.715767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba7ac8-7ae0-4d4e-afe8-741e5849bece admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:50.721722 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:50.722284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.722579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.722830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] PATH_INFO: `/v3/projects/4ff7468ed70c4d7f94053aa3167375e0` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.723987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff7468ed70c4d7f94053aa3167375e0)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:50.724256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:50.724988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.725431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.725707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] PATH_INFO: `/v3/users/0cac601e019745dc8e774c7dc689d57a` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.725927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.726877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cac601e019745dc8e774c7dc689d57a)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:50.728803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.772965 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff7468ed70c4d7f94053aa3167375e0', 'request_id': 'req-ec6f18b0-65ec-4505-84b8-9a008f60b028'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.776384 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cac601e019745dc8e774c7dc689d57a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.778701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff7468ed70c4d7f94053aa3167375e0', 'request_id': 'req-ec6f18b0-65ec-4505-84b8-9a008f60b028'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.784423 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cac601e019745dc8e774c7dc689d57a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.786285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff7468ed70c4d7f94053aa3167375e0', 'request_id': 'req-ec6f18b0-65ec-4505-84b8-9a008f60b028'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.791473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cac601e019745dc8e774c7dc689d57a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.792196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cac601e019745dc8e774c7dc689d57a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.792955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-1ccbe0db-7d5f-4d59-84a7-009ea4251253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cac601e019745dc8e774c7dc689d57a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.798348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] The token cache is being invalidate because project 4ff7468ed70c4d7f94053aa3167375e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:50.798537 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.799185 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-ec6f18b0-65ec-4505-84b8-9a008f60b028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:50.800657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1339/2657] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:50 2022] DELETE /identity/v3/projects/4ff7468ed70c4d7f94053aa3167375e0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:50.801854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71ee41-831e-4a3c-a0f6-2524cd103b71 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.802140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71ee41-831e-4a3c-a0f6-2524cd103b71 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.802416 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71ee41-831e-4a3c-a0f6-2524cd103b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.815619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1319/2658] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:50 2022] DELETE /identity/v3/users/0cac601e019745dc8e774c7dc689d57a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:50.817014 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59718057-8145-4332-9804-7d282ecef0ff None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.817323 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59718057-8145-4332-9804-7d282ecef0ff None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.817600 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59718057-8145-4332-9804-7d282ecef0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.828562 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9b71ee41-831e-4a3c-a0f6-2524cd103b71 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:50.829723 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9b71ee41-831e-4a3c-a0f6-2524cd103b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.840003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-59718057-8145-4332-9804-7d282ecef0ff None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:50.841234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-59718057-8145-4332-9804-7d282ecef0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.886010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:50.909352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1340/2659] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:50.910957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1320/2660] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:50.918950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a255cb-ac3e-45c3-a785-49a171d07e97 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.919306 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a255cb-ac3e-45c3-a785-49a171d07e97 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.919651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a255cb-ac3e-45c3-a785-49a171d07e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.927761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59718057-8145-4332-9804-7d282ecef0ff None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:50.930105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-59718057-8145-4332-9804-7d282ecef0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.941871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-71a255cb-ac3e-45c3-a785-49a171d07e97 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:50.943122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-71a255cb-ac3e-45c3-a785-49a171d07e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:50.952686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1341/2661] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:50.956592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a255cb-ac3e-45c3-a785-49a171d07e97 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:50.966332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a255cb-ac3e-45c3-a785-49a171d07e97 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:50.971377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:50.971934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:50.972160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:50.972430 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:50.973104 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:50.973507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:50.975049 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:50.990800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1342/2662] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:38:50.991920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.002640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3414-4916-4954-81ca-ed66adadbd95 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.005251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59718057-8145-4332-9804-7d282ecef0ff None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.008166 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.008853 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.009202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.009572 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.009712 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.010339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.010458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.010788 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.011133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.013307 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.020277 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:51.023511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.025123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.039230 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1321/2663] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:51.045313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1343/2664] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:50 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:51.048459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.058593 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e6b0-6130-4fe4-a9c7-e1ffbf9fe8d2 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.116324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.116943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.117203 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.117511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.118159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.120214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.147758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1322/2665] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:51.152162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.162883 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01889a-0352-4d7b-b199-b6518c6481e9 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.164601 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.166762 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.168702 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.169541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.169821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.170122 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.170781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.172382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.182561 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a6d0f-8165-44c0-8c4d-32a998d7ba8e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.184350 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1344/2666] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:51.189312 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.193357 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.194149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.194536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.194946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] PATH_INFO: `/v3/projects/a6045c3f4a0f430196258e295cf3efce` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.198447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6045c3f4a0f430196258e295cf3efce)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.200911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.212494 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41415584-b3e5-488e-95da-2859db7f1456 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.217386 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.218007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.218284 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.218556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.219612 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.221579 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.233544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1345/2667] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:51.237689 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.250575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f91d1-95dd-4e24-b2fd-ff9e2df7a2d1 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.259322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6045c3f4a0f430196258e295cf3efce', 'request_id': 'req-71f4630c-d351-4fc3-a622-8ade847680f5'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.265498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6045c3f4a0f430196258e295cf3efce', 'request_id': 'req-71f4630c-d351-4fc3-a622-8ade847680f5'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.274681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6045c3f4a0f430196258e295cf3efce', 'request_id': 'req-71f4630c-d351-4fc3-a622-8ade847680f5'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.288350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] The token cache is being invalidate because project a6045c3f4a0f430196258e295cf3efce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:51.288809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.289597 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-71f4630c-d351-4fc3-a622-8ade847680f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.291244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1323/2668] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:51 2022] DELETE /identity/v3/projects/a6045c3f4a0f430196258e295cf3efce => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:51.296748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa32fc-5380-4b3b-90e1-97ea80bd76ae None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.297085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa32fc-5380-4b3b-90e1-97ea80bd76ae None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.297563 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa32fc-5380-4b3b-90e1-97ea80bd76ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.318766 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8682835-4df3-4035-bec8-7de0e886f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.319701 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682835-4df3-4035-bec8-7de0e886f4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.320141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682835-4df3-4035-bec8-7de0e886f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.320528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682835-4df3-4035-bec8-7de0e886f4f5 admin admin] PATH_INFO: `/v3/projects/3a8b139d07224c3482f8c191a2846be9/users/70b4883fd1f64b3cabb7f5c39c78e364/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.320891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-13aa32fc-5380-4b3b-90e1-97ea80bd76ae None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:51.322142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-13aa32fc-5380-4b3b-90e1-97ea80bd76ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.331042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1324/2669] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:51.338673 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1910f4-7c95-41b3-bfa0-e1b201802598 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.338960 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1910f4-7c95-41b3-bfa0-e1b201802598 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.339238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1910f4-7c95-41b3-bfa0-e1b201802598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.340871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8682835-4df3-4035-bec8-7de0e886f4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a8b139d07224c3482f8c191a2846be9, user_id=70b4883fd1f64b3cabb7f5c39c78e364, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.344914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8682835-4df3-4035-bec8-7de0e886f4f5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.360441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8b1910f4-7c95-41b3-bfa0-e1b201802598 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:51.361713 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8b1910f4-7c95-41b3-bfa0-e1b201802598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.373109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1346/2670] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:51 2022] PUT /identity/v3/projects/3a8b139d07224c3482f8c191a2846be9/users/70b4883fd1f64b3cabb7f5c39c78e364/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:51.375781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.375905 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.376552 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.408475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] MFA Rules not processed for user `70b4883fd1f64b3cabb7f5c39c78e364`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:51.409892 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.429968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1325/2671] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:51.434366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1910f4-7c95-41b3-bfa0-e1b201802598 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.440412 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:51.445468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1910f4-7c95-41b3-bfa0-e1b201802598 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.452821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.455653 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.455940 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.456208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.456881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:51.458598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.461738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.479025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1326/2672] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:51.483651 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.489290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:51.497848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1347/2673] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:51.499463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.500781 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7604-c025-4ed5-b368-6231f26fb15b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.506148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.506592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.509295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.509295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.509295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.509295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.511017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.526261 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29a50a-f489-482f-9020-28f4c88577f1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.532586 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.533582 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.534002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.534555 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.535474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.537821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.543806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1327/2674] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:51.549043 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.590995 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1348/2675] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:51.599348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.672994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cd88c-f547-4666-bae8-e997bd2d01f6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.708143 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798f2cc-7a9a-4e88-86b2-64b462d0df83 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.715470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.716389 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.716630 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.717673 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.719020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.722047 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.748157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1349/2676] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:51.754575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.757723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.758471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.758662 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.758916 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.759592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.761497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.771456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbbca2-f7fd-476e-8e96-3f5b0a4065b8 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.782832 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.783449 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.783727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.783997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.784644 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.786260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.794405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1328/2677] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:51 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:51.794405 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.796445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.800109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1350/2678] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:51.803842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.811532 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d90d19-b2e0-4db2-bd4a-548fd2146214 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.813890 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09133105-d437-4a5e-8902-546ecfaf3eda admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.819266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.819736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.819924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.820229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.820514 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] PATH_INFO: `/v3/projects/03da2be4f7334fa2a74c281b64e1be21` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.821401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.821872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.822061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03da2be4f7334fa2a74c281b64e1be21)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.822323 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.823237 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.824520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.826793 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.838949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1351/2679] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:51.839704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.850497 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce7e01-a4b4-4561-b3b6-5e67dfd442dc admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.855128 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.855829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.856171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.856511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] PATH_INFO: `/v3/projects/11e65ea61b4c4089a4348f07ab4234a3/users/78277a0de4c747c7a2b2c5839555f78c/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.871732 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e65ea61b4c4089a4348f07ab4234a3, user_id=78277a0de4c747c7a2b2c5839555f78c, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.874571 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.878506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03da2be4f7334fa2a74c281b64e1be21', 'request_id': 'req-380e6443-cb2d-44b9-885f-527a32701c12'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.884147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03da2be4f7334fa2a74c281b64e1be21', 'request_id': 'req-380e6443-cb2d-44b9-885f-527a32701c12'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.886041 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:51.888283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1352/2680] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:51 2022] PUT /identity/v3/projects/11e65ea61b4c4089a4348f07ab4234a3/users/78277a0de4c747c7a2b2c5839555f78c/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:51.889118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.895390 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03da2be4f7334fa2a74c281b64e1be21', 'request_id': 'req-380e6443-cb2d-44b9-885f-527a32701c12'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.909798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] The token cache is being invalidate because project 03da2be4f7334fa2a74c281b64e1be21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:51.910087 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.910737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:51.912237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1329/2681] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:51 2022] DELETE /identity/v3/projects/03da2be4f7334fa2a74c281b64e1be21 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:51.913068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.915941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.951933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f3376-a781-40c4-9cee-934f45953e12 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.956560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.957154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.957474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.957752 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.958418 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.960018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.971362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1353/2682] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:51.974410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:51.976521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:51.987434 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e6443-cb2d-44b9-885f-527a32701c12 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.990118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbb75b-cfee-4f08-aefb-0335a2d02dcb admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:51.992063 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.993135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.993451 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.993727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.994371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.994714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:51.995326 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:51.995600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:51.995869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] PATH_INFO: `/v3/users/ac31db49176f4bb8a733e5ba9d8654e9` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:51.995964 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:51.996977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac31db49176f4bb8a733e5ba9d8654e9)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:51.998991 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.006276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1330/2683] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:52.007038 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:52.008796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.012451 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:52.022242 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8363e8-9ed5-4ca2-b3d2-6cc879595537 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:52.027123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:52.027780 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.028066 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.028347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] PATH_INFO: `/v3/projects/8ca8327ae5d34d9ea449c0cd9ab8caf9/users/33f58b535a004405a39a1a031c84967b/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.045009 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca8327ae5d34d9ea449c0cd9ab8caf9, user_id=33f58b535a004405a39a1a031c84967b, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:52.047834 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.053642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac31db49176f4bb8a733e5ba9d8654e9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.061775 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac31db49176f4bb8a733e5ba9d8654e9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.062315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1331/2684] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:52 2022] PUT /identity/v3/projects/8ca8327ae5d34d9ea449c0cd9ab8caf9/users/33f58b535a004405a39a1a031c84967b/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:52.062501 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac31db49176f4bb8a733e5ba9d8654e9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.063271 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac31db49176f4bb8a733e5ba9d8654e9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.063454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:52.065365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.084737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-19f0c179-2e0b-4af0-ab4e-2715a7731c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac31db49176f4bb8a733e5ba9d8654e9'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.118953 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1354/2685] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:51 2022] DELETE /identity/v3/users/ac31db49176f4bb8a733e5ba9d8654e9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:52.126069 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.126563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.127018 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.172807 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a2523-5446-423a-998f-929278cd12a6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:52.195946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] MFA Rules not processed for user `33f58b535a004405a39a1a031c84967b`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:52.198668 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.260208 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89175803-54c7-4ae4-b270-51bc282f4b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:52.261686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89175803-54c7-4ae4-b270-51bc282f4b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.262396 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89175803-54c7-4ae4-b270-51bc282f4b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.262939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89175803-54c7-4ae4-b270-51bc282f4b6e admin admin] PATH_INFO: `/v3/projects/11e65ea61b4c4089a4348f07ab4234a3/users/78277a0de4c747c7a2b2c5839555f78c/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.267581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89175803-54c7-4ae4-b270-51bc282f4b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e65ea61b4c4089a4348f07ab4234a3, user_id=78277a0de4c747c7a2b2c5839555f78c, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:52.274678 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89175803-54c7-4ae4-b270-51bc282f4b6e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.289224 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:52.299248 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1355/2686] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:52.299571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1332/2687] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:52 2022] PUT /identity/v3/projects/11e65ea61b4c4089a4348f07ab4234a3/users/78277a0de4c747c7a2b2c5839555f78c/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:52.308085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:52.309618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.314550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.315046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.315469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.363520 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] MFA Rules not processed for user `78277a0de4c747c7a2b2c5839555f78c`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:52.364758 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.392794 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4604a4d-769c-4376-ab03-53eb51d0de39 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:52.436915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:52.459797 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:52.460413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.460693 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.460968 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.469585 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1333/2688] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:52.481256 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:52.483164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.501655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf948c8-4ebe-4943-8a43-d2045671989e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:52.508376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:52.509599 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.509882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.510410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.521837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:52.523362 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.533980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1334/2689] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:52.534785 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:52.535447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:52.536155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.537035 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.553414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1356/2690] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:52.618533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63068f-3245-4188-bae1-71fcaf4aedad service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:52.624092 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:52.625534 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.625813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.626081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] PATH_INFO: `/v3/projects/e4bcbe058eb14823ad96fe547313cfee` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.627391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4bcbe058eb14823ad96fe547313cfee)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:52.629413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.632250 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:52.641827 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ae2f9-22f4-4a3b-9f60-dd7fe19ee48c service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:52.646806 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af6fe7-aa23-4b1f-9750-56e276607d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:52.647411 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af6fe7-aa23-4b1f-9750-56e276607d70 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.647872 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af6fe7-aa23-4b1f-9750-56e276607d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.648144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af6fe7-aa23-4b1f-9750-56e276607d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.650596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-90af6fe7-aa23-4b1f-9750-56e276607d70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.683587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4bcbe058eb14823ad96fe547313cfee', 'request_id': 'req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.691994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4bcbe058eb14823ad96fe547313cfee', 'request_id': 'req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.704234 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4bcbe058eb14823ad96fe547313cfee', 'request_id': 'req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.724761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] The token cache is being invalidate because project e4bcbe058eb14823ad96fe547313cfee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:52.725103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.725858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-0ea92b09-8d00-419c-b1bb-27c1a445b8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:52.731413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1335/2691] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:52 2022] DELETE /identity/v3/projects/e4bcbe058eb14823ad96fe547313cfee => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:52.733414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0d63b-8b2a-4732-94ae-f3833f4c1454 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.733760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0d63b-8b2a-4732-94ae-f3833f4c1454 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.734185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0d63b-8b2a-4732-94ae-f3833f4c1454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.737045 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af6fe7-aa23-4b1f-9750-56e276607d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:52.739878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af6fe7-aa23-4b1f-9750-56e276607d70 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.743095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:52.758447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-84f0d63b-8b2a-4732-94ae-f3833f4c1454 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:52.761441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-84f0d63b-8b2a-4732-94ae-f3833f4c1454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.767214 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1357/2692] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:52.854796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1336/2693] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:52.863053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087ba6f-fc56-43ff-a323-a63a6f0202f1 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.863339 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087ba6f-fc56-43ff-a323-a63a6f0202f1 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.863666 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087ba6f-fc56-43ff-a323-a63a6f0202f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.909030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2087ba6f-fc56-43ff-a323-a63a6f0202f1 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:52.910399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2087ba6f-fc56-43ff-a323-a63a6f0202f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:52.925593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1358/2694] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:52.931137 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0d63b-8b2a-4732-94ae-f3833f4c1454 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:52.963222 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0d63b-8b2a-4732-94ae-f3833f4c1454 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:52.969361 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:52.969973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:52.970282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:52.970889 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:52.971544 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:52.971846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:52.973629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:52.987524 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1337/2695] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:52.991561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087ba6f-fc56-43ff-a323-a63a6f0202f1 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.001521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087ba6f-fc56-43ff-a323-a63a6f0202f1 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.006467 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.007137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.007415 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.007711 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.008367 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.010317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.037978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1359/2696] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:52 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:53.042119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.053677 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3f28-1e84-4338-a3e7-2b729170d974 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.078671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.080946 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.112178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.112816 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.113106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.113414 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.114050 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.115987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.137424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1338/2697] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:53 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:53.141449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.145406 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d0609-b70e-4c49-9cc9-7c3e63194114 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.149982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.150600 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.150874 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.151147 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] PATH_INFO: `/v3/users/70b4883fd1f64b3cabb7f5c39c78e364` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.151645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa07fe8-2ee4-4e99-9c77-b694c24cd1b6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.152377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b4883fd1f64b3cabb7f5c39c78e364)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.154967 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.156493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:53.156849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.157447 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.157943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.158074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.158613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.160269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.174348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1339/2698] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:53.175566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.185068 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668f708-75d2-4790-9a9f-a85c357d45b3 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.191084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.191747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.192267 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.192553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.193196 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.194957 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.205013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1340/2699] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:53.207305 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b4883fd1f64b3cabb7f5c39c78e364'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.209168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.214921 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b4883fd1f64b3cabb7f5c39c78e364'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.215623 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b4883fd1f64b3cabb7f5c39c78e364'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.216247 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b4883fd1f64b3cabb7f5c39c78e364'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.219992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ceff2c-4fcf-4e1a-9d21-fdb33999cd6f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.224081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-581b07d7-b89a-4b1f-a17b-e6635a9fdf60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b4883fd1f64b3cabb7f5c39c78e364'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.225296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e47e10-0ae9-427b-b26c-e12934f70dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.225933 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e47e10-0ae9-427b-b26c-e12934f70dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.226214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e47e10-0ae9-427b-b26c-e12934f70dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.226522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e47e10-0ae9-427b-b26c-e12934f70dd5 admin admin] PATH_INFO: `/v3/projects/4ef9e7da46dd4ed3af296578adc16e62/users/2b8e6469338b4490b102bbb053bd0978/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.241665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1360/2700] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:53 2022] DELETE /identity/v3/users/70b4883fd1f64b3cabb7f5c39c78e364 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:53.241905 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e47e10-0ae9-427b-b26c-e12934f70dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef9e7da46dd4ed3af296578adc16e62, user_id=2b8e6469338b4490b102bbb053bd0978, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.246010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.246299 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.246578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.247169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e47e10-0ae9-427b-b26c-e12934f70dd5 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.260484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1341/2701] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:53 2022] PUT /identity/v3/projects/4ef9e7da46dd4ed3af296578adc16e62/users/2b8e6469338b4490b102bbb053bd0978/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:53.270422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:53.270779 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.271291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.271715 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.271821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.299491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] MFA Rules not processed for user `2b8e6469338b4490b102bbb053bd0978`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:53.300722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.322304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:53.339480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1361/2702] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:53.346433 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.347919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.373554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1342/2703] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:53.384679 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.386090 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.405648 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44dfef-c675-432c-b683-1b77feae1bed None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.409590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.410286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.410573 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.411109 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.422731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.424205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.433601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1362/2704] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:53.444448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea5fb6-dd11-4d9a-9cc3-fb97986191cf None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.448313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca0d2e-cc54-43f2-a96b-535bf4d1a981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.448913 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0d2e-cc54-43f2-a96b-535bf4d1a981 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.449193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0d2e-cc54-43f2-a96b-535bf4d1a981 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.449490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0d2e-cc54-43f2-a96b-535bf4d1a981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.458375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca0d2e-cc54-43f2-a96b-535bf4d1a981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.459795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca0d2e-cc54-43f2-a96b-535bf4d1a981 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.469058 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1343/2705] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:53.490124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.491667 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.505659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa95ab8-bd7c-4e5e-a0d9-613f7fad1238 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.510311 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.510935 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.511216 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.511489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] PATH_INFO: `/v3/users/2b8e6469338b4490b102bbb053bd0978` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.512620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8e6469338b4490b102bbb053bd0978)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.515056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.532934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:53.570570 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8e6469338b4490b102bbb053bd0978'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.598067 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8e6469338b4490b102bbb053bd0978'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.599490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8e6469338b4490b102bbb053bd0978'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.600242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8e6469338b4490b102bbb053bd0978'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.609352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8e6469338b4490b102bbb053bd0978'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.630641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1363/2706] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:53 2022] DELETE /identity/v3/users/2b8e6469338b4490b102bbb053bd0978 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:53.636207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.636721 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.636823 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.660142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:53.661458 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.728903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1344/2707] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:53.740530 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.742102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.742236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.748739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.762354 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2487-1ade-47a0-8073-cbfc501317c2 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.768766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.770029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.770178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.770495 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] PATH_INFO: `/v3/projects/3a8b139d07224c3482f8c191a2846be9` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.771871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a8b139d07224c3482f8c191a2846be9)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.774484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.836057 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dc1e4-4d24-4890-909f-17454c2e9ba9 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.842497 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a8b139d07224c3482f8c191a2846be9', 'request_id': 'req-34fb1424-5a55-4e38-a843-2d658d3ea655'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.849472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a8b139d07224c3482f8c191a2846be9', 'request_id': 'req-34fb1424-5a55-4e38-a843-2d658d3ea655'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.851977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:53.859521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a8b139d07224c3482f8c191a2846be9', 'request_id': 'req-34fb1424-5a55-4e38-a843-2d658d3ea655'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.873655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] The token cache is being invalidate because project 3a8b139d07224c3482f8c191a2846be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:53.873987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.874642 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:53.876171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1345/2708] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:53 2022] DELETE /identity/v3/projects/3a8b139d07224c3482f8c191a2846be9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:53.879966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:53.881348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.893056 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.893833 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.894419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.894714 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.896592 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:53.963810 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1424-5a55-4e38-a843-2d658d3ea655 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:53.968924 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:53.969699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:53.970082 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:53.970428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] PATH_INFO: `/v3/users/78277a0de4c747c7a2b2c5839555f78c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:53.972803 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78277a0de4c747c7a2b2c5839555f78c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.974502 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.975949 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:53.977419 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:53.983499 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:53.992304 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1364/2709] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:54.052274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78277a0de4c747c7a2b2c5839555f78c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.060462 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78277a0de4c747c7a2b2c5839555f78c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.072266 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78277a0de4c747c7a2b2c5839555f78c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.073160 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78277a0de4c747c7a2b2c5839555f78c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.073946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78277a0de4c747c7a2b2c5839555f78c'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.095282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1346/2710] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:38:53 2022] DELETE /identity/v3/users/78277a0de4c747c7a2b2c5839555f78c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:54.474926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:54.476551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:54.488597 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7355e-9e4d-4dae-88bc-4e114bdf9f10 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:54.548206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:54.550344 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:54.567245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:54.567989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.568333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.568671 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] PATH_INFO: `/v3/projects/4ef9e7da46dd4ed3af296578adc16e62` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.570159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef9e7da46dd4ed3af296578adc16e62)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:54.571918 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:54.630894 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef9e7da46dd4ed3af296578adc16e62', 'request_id': 'req-3de07071-2281-4376-aa5d-d841c7dfe27c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.633846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6069aab-5432-494c-970f-45656ff8cd08 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:54.636525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef9e7da46dd4ed3af296578adc16e62', 'request_id': 'req-3de07071-2281-4376-aa5d-d841c7dfe27c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.645548 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef9e7da46dd4ed3af296578adc16e62', 'request_id': 'req-3de07071-2281-4376-aa5d-d841c7dfe27c'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.646797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:54.659364 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] The token cache is being invalidate because project 4ef9e7da46dd4ed3af296578adc16e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:54.659659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.660351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-3de07071-2281-4376-aa5d-d841c7dfe27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.662479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1365/2711] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:54 2022] DELETE /identity/v3/projects/4ef9e7da46dd4ed3af296578adc16e62 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:54.667083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97742819-5b81-4f2d-b066-801e6f66e9a5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.667438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97742819-5b81-4f2d-b066-801e6f66e9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.667803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97742819-5b81-4f2d-b066-801e6f66e9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.687102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-97742819-5b81-4f2d-b066-801e6f66e9a5 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:54.688296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-97742819-5b81-4f2d-b066-801e6f66e9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:54.696198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:54.696903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.697186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.697503 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] PATH_INFO: `/v3/projects/11e65ea61b4c4089a4348f07ab4234a3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.698638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e65ea61b4c4089a4348f07ab4234a3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:54.700559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:54.747878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e65ea61b4c4089a4348f07ab4234a3', 'request_id': 'req-c4574915-d526-49d0-a2e2-5ca2dfd85a82'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.748982 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1366/2712] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:54.754012 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e65ea61b4c4089a4348f07ab4234a3', 'request_id': 'req-c4574915-d526-49d0-a2e2-5ca2dfd85a82'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.757754 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694da0f-61e2-4fc7-b361-376c661f5536 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.758071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694da0f-61e2-4fc7-b361-376c661f5536 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.758405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694da0f-61e2-4fc7-b361-376c661f5536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.762292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e65ea61b4c4089a4348f07ab4234a3', 'request_id': 'req-c4574915-d526-49d0-a2e2-5ca2dfd85a82'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.773978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] The token cache is being invalidate because project 11e65ea61b4c4089a4348f07ab4234a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:38:54.774263 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.774881 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-c4574915-d526-49d0-a2e2-5ca2dfd85a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:38:54.776366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1347/2713] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:38:54 2022] DELETE /identity/v3/projects/11e65ea61b4c4089a4348f07ab4234a3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:54.778580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-3694da0f-61e2-4fc7-b361-376c661f5536 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:54.779893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-3694da0f-61e2-4fc7-b361-376c661f5536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:54.780962 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e680c1-8f71-458d-8c36-6b41a9587a19 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.781248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e680c1-8f71-458d-8c36-6b41a9587a19 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.781551 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e680c1-8f71-458d-8c36-6b41a9587a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.800202 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f0e680c1-8f71-458d-8c36-6b41a9587a19 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:54.801350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f0e680c1-8f71-458d-8c36-6b41a9587a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:54.805179 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:54.842842 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1367/2714] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:54.854285 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694da0f-61e2-4fc7-b361-376c661f5536 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:54.864631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694da0f-61e2-4fc7-b361-376c661f5536 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:54.864733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1348/2715] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:54.869176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:54.869810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.870095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.870361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.871021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:54.871324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:54.872282 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac3903-ec8a-469e-8494-238974f98485 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.872588 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac3903-ec8a-469e-8494-238974f98485 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.872868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac3903-ec8a-469e-8494-238974f98485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.872992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:54.885296 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1368/2716] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:38:54.890141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:54.893639 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-1aac3903-ec8a-469e-8494-238974f98485 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:54.894859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-1aac3903-ec8a-469e-8494-238974f98485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:54.899924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1048cf-6b04-46a6-852d-d1875499a86c admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:54.902841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1349/2717] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:54.904483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:54.905099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.905378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.905652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.906270 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:54.906879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac3903-ec8a-469e-8494-238974f98485 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:54.908158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:54.915849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac3903-ec8a-469e-8494-238974f98485 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:54.920431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:54.921013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:54.921311 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:54.921587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:54.922223 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:38:54.922528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:54.924110 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:54.939365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1369/2718] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:54 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:54.939557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1350/2719] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:38:54.940436 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:54.940813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:54.950265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f005612-4c31-4a26-b864-c933ab43af1e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:54.975510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:55.004456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f8a4f-d351-414f-b357-66eb4f015ae2 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.009115 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.009297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.009746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.009858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.010016 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.010116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.010277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.010383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.010892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.011006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.012782 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.012917 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.037298 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1370/2720] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:54 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:55.039027 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.045526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1351/2721] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:54 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:55.046714 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.049024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b164a-53fe-4c4f-901b-b24fc8cc2dc9 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.104245 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.104802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.105086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.105396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.106037 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.108160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.110917 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a8603-7eaa-42ab-92ff-f6230a0b460e admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.115587 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.116185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.116461 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.116731 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.117378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.117920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1371/2722] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:55.119292 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.121902 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.132092 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a1d19-7bcf-40ec-8822-2f287332f201 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.136945 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.137565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.137845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.138118 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.138778 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.140361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.144053 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1352/2723] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:55 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:38:55.145128 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.151914 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1372/2724] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:55.155930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.156980 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c3cb7-30e5-4add-96b5-f80a34dde488 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.161747 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.162366 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.162641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.162908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.163535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.165142 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.165343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549aafb-7e1a-416a-a3e7-76f03da3dec5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.169928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.171038 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.171314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.171585 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] PATH_INFO: `/v3/projects/afa4735b7c5b46bc9a0ca81f4733de00/users/9d593e55f72d4757886261b383e1794f/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.175505 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1353/2725] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:55.179296 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.179651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:55.188073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa4735b7c5b46bc9a0ca81f4733de00, user_id=9d593e55f72d4757886261b383e1794f, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.190790 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.190910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e5ba-b1fa-4961-8ca8-2059e02afbb0 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.195975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b308c08-0c96-4db7-a485-1bdcad4e8544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.196568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b308c08-0c96-4db7-a485-1bdcad4e8544 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.196869 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b308c08-0c96-4db7-a485-1bdcad4e8544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.197291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b308c08-0c96-4db7-a485-1bdcad4e8544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.197938 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b308c08-0c96-4db7-a485-1bdcad4e8544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.199566 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b308c08-0c96-4db7-a485-1bdcad4e8544 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.203522 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1373/2726] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:55 2022] PUT /identity/v3/projects/afa4735b7c5b46bc9a0ca81f4733de00/users/9d593e55f72d4757886261b383e1794f/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:38:55.210172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1354/2727] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:55.214456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.265911 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.266251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.266468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.286376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:55.288554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:55.290676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0484bb-2aa9-4b42-89d4-0e188497d302 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.296583 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e54c0-ff82-448b-87f2-08a9a15ef8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.297238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1355/2728] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:55.297607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e54c0-ff82-448b-87f2-08a9a15ef8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.300933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e54c0-ff82-448b-87f2-08a9a15ef8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.300933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e54c0-ff82-448b-87f2-08a9a15ef8b2 admin admin] PATH_INFO: `/v3/projects/636f8bc0692549f2aa7b000f82149231/users/414d2a5e741149648268b3d0c1eef021/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.306730 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.308607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:55.324252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e54c0-ff82-448b-87f2-08a9a15ef8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636f8bc0692549f2aa7b000f82149231, user_id=414d2a5e741149648268b3d0c1eef021, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.328952 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e54c0-ff82-448b-87f2-08a9a15ef8b2 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.346445 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1374/2729] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:55 2022] PUT /identity/v3/projects/636f8bc0692549f2aa7b000f82149231/users/414d2a5e741149648268b3d0c1eef021/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:55.351510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.352641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.353006 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.375076 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea3b9-b86c-42b3-a05d-c9a82a4fdc4e None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.379463 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:55.380930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:55.407123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:55.435824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f5279-68cb-4536-acf2-7c77e45a9ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.436557 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f5279-68cb-4536-acf2-7c77e45a9ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.436939 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f5279-68cb-4536-acf2-7c77e45a9ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.437359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f5279-68cb-4536-acf2-7c77e45a9ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.445212 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1375/2730] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:55.455290 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:55.456712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:55.469172 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f8907-e588-4065-8f66-63a39de7ab6b None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:55.473156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4c91f-9efd-4dd3-9fa6-3f92badabc89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:55.473785 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4c91f-9efd-4dd3-9fa6-3f92badabc89 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:55.474088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4c91f-9efd-4dd3-9fa6-3f92badabc89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:55.474361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4c91f-9efd-4dd3-9fa6-3f92badabc89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:55.483984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4c91f-9efd-4dd3-9fa6-3f92badabc89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.485489 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4c91f-9efd-4dd3-9fa6-3f92badabc89 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.494950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1376/2731] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:55.499523 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f5279-68cb-4536-acf2-7c77e45a9ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:55.501044 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f5279-68cb-4536-acf2-7c77e45a9ed9 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:55.510681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1356/2732] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:56.160929 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b233-160f-4cd0-95b4-656671cd5f6c None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.161475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b233-160f-4cd0-95b4-656671cd5f6c None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.161761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b233-160f-4cd0-95b4-656671cd5f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.175744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.175744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.175744 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.199978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-37d0b233-160f-4cd0-95b4-656671cd5f6c None None] MFA Rules not processed for user `414d2a5e741149648268b3d0c1eef021`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:56.201421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-37d0b233-160f-4cd0-95b4-656671cd5f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:56.206084 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] MFA Rules not processed for user `9d593e55f72d4757886261b383e1794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:56.208008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:56.261803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:56.286675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1377/2733] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:56.291252 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1357/2734] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:38:56.294776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b233-160f-4cd0-95b4-656671cd5f6c None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:56.303397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:56.303547 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b233-160f-4cd0-95b4-656671cd5f6c None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:56.305059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:56.309761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:56.310387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.311359 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.311633 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.312266 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:56.314227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:56.354318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1378/2735] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:38:56 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:56.358525 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:56.392565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13858a-77f4-4b87-b2f0-211937da27ca None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:56.414604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:56.431595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e360e-25cd-44f7-92a0-e10d305676d5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:56.438486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:56.439280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.439410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.439635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.440422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:56.443618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:56.458304 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:56.458932 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.459209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.459488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.468604 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1379/2736] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:38:56 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:38:56.472414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:56.485826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7f35d-d99f-465d-963a-3920447b33ea admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:56.490934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:56.492229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.492584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.492925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.493710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:56.495631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:56.512189 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1380/2737] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:38:56.513561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:56.527713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d510de2-5a5d-47a4-8975-c9cc284da8fe admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:56.536160 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:56.537022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.537336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.537685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] PATH_INFO: `/v3/projects/6e14f60e46df4243b7d9b78ab67c4856/users/acb79c6d46e547a88a2aefe78b22c0ee/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.556421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e14f60e46df4243b7d9b78ab67c4856, user_id=acb79c6d46e547a88a2aefe78b22c0ee, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:56.557856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:56.559399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:56.559540 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:56.574109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1358/2738] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:38:56.579460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1381/2739] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:56 2022] PUT /identity/v3/projects/6e14f60e46df4243b7d9b78ab67c4856/users/acb79c6d46e547a88a2aefe78b22c0ee/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:56.583174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:56.600407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2717945-2a17-4dd2-b013-fa58107bf5a3 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:56.668454 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687810c-4419-4583-bc0b-886678c272ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:56.669260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687810c-4419-4583-bc0b-886678c272ca admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.669571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687810c-4419-4583-bc0b-886678c272ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.670033 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687810c-4419-4583-bc0b-886678c272ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.671416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687810c-4419-4583-bc0b-886678c272ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:56.673415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687810c-4419-4583-bc0b-886678c272ca admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:56.686655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1359/2740] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:38:56.690900 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:56.704593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118ad52-3913-4f37-8e2c-dd88fb82c71a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:56.709642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:56.710341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:56.710591 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:56.710864 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] PATH_INFO: `/v3/projects/6e14f60e46df4243b7d9b78ab67c4856/users/acb79c6d46e547a88a2aefe78b22c0ee/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:56.712836 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e14f60e46df4243b7d9b78ab67c4856, user_id=acb79c6d46e547a88a2aefe78b22c0ee, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:56.715568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:56.727805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1382/2741] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:38:56 2022] PUT /identity/v3/projects/6e14f60e46df4243b7d9b78ab67c4856/users/acb79c6d46e547a88a2aefe78b22c0ee/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:38:57.380699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037f8e6-029e-42eb-b3b8-3a64051e36d0 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:57.381165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037f8e6-029e-42eb-b3b8-3a64051e36d0 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:57.381306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037f8e6-029e-42eb-b3b8-3a64051e36d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:57.416603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-a037f8e6-029e-42eb-b3b8-3a64051e36d0 None None] MFA Rules not processed for user `acb79c6d46e547a88a2aefe78b22c0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:57.418815 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a037f8e6-029e-42eb-b3b8-3a64051e36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:57.500646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:57.504572 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1360/2742] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:57.516136 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:57.589295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d2f86-7160-4691-b449-c3e2ab5b1a38 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:57.598758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:57.599469 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:57.599811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:57.600150 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:57.620512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:38:57.662516 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:57.663990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:57.674475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1383/2743] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:58.035682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217d06-ac37-4032-932e-f67b476e42b8 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:58.035968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217d06-ac37-4032-932e-f67b476e42b8 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:58.036243 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217d06-ac37-4032-932e-f67b476e42b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:58.055723 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-10217d06-ac37-4032-932e-f67b476e42b8 None None] MFA Rules not processed for user `9d593e55f72d4757886261b383e1794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:38:58.056954 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-10217d06-ac37-4032-932e-f67b476e42b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:58.140029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1361/2744] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:38:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:38:59.100907 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:59.102371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:59.114821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aab2b0-55a1-4112-8dfa-a1c6f9351838 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:59.120309 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:59.121129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:59.121450 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:59.121881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:59.136924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:59.139546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:59.158333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1384/2745] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:38:59.264761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217d06-ac37-4032-932e-f67b476e42b8 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:38:59.341313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217d06-ac37-4032-932e-f67b476e42b8 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:38:59.346382 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85dd17-dcf7-415f-821c-dc2017c51694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:38:59.347053 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85dd17-dcf7-415f-821c-dc2017c51694 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:38:59.347336 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85dd17-dcf7-415f-821c-dc2017c51694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:38:59.347611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85dd17-dcf7-415f-821c-dc2017c51694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:38:59.349536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-bd85dd17-dcf7-415f-821c-dc2017c51694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:38:59.422989 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85dd17-dcf7-415f-821c-dc2017c51694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:38:59.424438 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85dd17-dcf7-415f-821c-dc2017c51694 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:38:59.434235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1362/2746] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:38:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:00.332807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:00.334742 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:00.415230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c48f8-31bd-40dd-8f7c-6746dedba789 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:00.421260 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:00.422373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:00.424356 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:00.426017 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] PATH_INFO: `/v3/users/33f58b535a004405a39a1a031c84967b` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:00.428717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f58b535a004405a39a1a031c84967b)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:00.432615 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:00.499724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f58b535a004405a39a1a031c84967b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:00.512320 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f58b535a004405a39a1a031c84967b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:00.513283 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f58b535a004405a39a1a031c84967b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:00.514197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f58b535a004405a39a1a031c84967b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:00.526355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f58b535a004405a39a1a031c84967b'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:00.562130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1385/2747] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:00 2022] DELETE /identity/v3/users/33f58b535a004405a39a1a031c84967b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:00.566698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:00.567211 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:00.567300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:00.603279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:00.604733 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:00.696005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:00.718315 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1363/2748] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:00.759360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:00.760856 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:00.798652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:00.850611 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291974a3-9773-479d-bc31-50567f8bf3e7 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:00.854824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621127a9-b088-4a90-a4ee-dcf215779674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:00.855461 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621127a9-b088-4a90-a4ee-dcf215779674 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:00.856088 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621127a9-b088-4a90-a4ee-dcf215779674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:00.856272 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621127a9-b088-4a90-a4ee-dcf215779674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:00.867869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621127a9-b088-4a90-a4ee-dcf215779674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:00.869896 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621127a9-b088-4a90-a4ee-dcf215779674 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:00.881908 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1386/2749] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:01.612321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:01.613861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:01.627254 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2212cab-dbcb-4398-bfe4-73c9ab063ddf None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:01.632700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:01.633664 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:01.634005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:01.634345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] PATH_INFO: `/v3/projects/8ca8327ae5d34d9ea449c0cd9ab8caf9` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:01.635553 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca8327ae5d34d9ea449c0cd9ab8caf9)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:01.637872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:01.709177 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca8327ae5d34d9ea449c0cd9ab8caf9', 'request_id': 'req-79b380b1-8037-409c-9f25-3bced212dff1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:01.717681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca8327ae5d34d9ea449c0cd9ab8caf9', 'request_id': 'req-79b380b1-8037-409c-9f25-3bced212dff1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:01.727831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca8327ae5d34d9ea449c0cd9ab8caf9', 'request_id': 'req-79b380b1-8037-409c-9f25-3bced212dff1'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:01.745060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] The token cache is being invalidate because project 8ca8327ae5d34d9ea449c0cd9ab8caf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:01.745522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:01.746628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-79b380b1-8037-409c-9f25-3bced212dff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:01.748893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1364/2750] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:01 2022] DELETE /identity/v3/projects/8ca8327ae5d34d9ea449c0cd9ab8caf9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:01.753295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2cab-f9cc-4df7-9a54-d100faa5a77d None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:01.753795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2cab-f9cc-4df7-9a54-d100faa5a77d None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:01.754013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2cab-f9cc-4df7-9a54-d100faa5a77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:01.773878 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-28cd2cab-f9cc-4df7-9a54-d100faa5a77d None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:01.775410 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-28cd2cab-f9cc-4df7-9a54-d100faa5a77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:01.849632 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1387/2751] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:01.852973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa441b-cdfc-4a0f-8f0f-119b1b9344ee None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:01.853261 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa441b-cdfc-4a0f-8f0f-119b1b9344ee None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:01.853560 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa441b-cdfc-4a0f-8f0f-119b1b9344ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:01.880920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-fbaa441b-cdfc-4a0f-8f0f-119b1b9344ee None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:01.882330 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-fbaa441b-cdfc-4a0f-8f0f-119b1b9344ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:01.891998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1365/2752] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:01.895980 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd2cab-f9cc-4df7-9a54-d100faa5a77d None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:01.914087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd2cab-f9cc-4df7-9a54-d100faa5a77d None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:01.919786 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:01.920925 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:01.921202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:01.921593 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:01.923329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:39:01.923329 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:01.924368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:01.943513 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1388/2753] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:39:01.945787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa441b-cdfc-4a0f-8f0f-119b1b9344ee None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:01.957820 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa441b-cdfc-4a0f-8f0f-119b1b9344ee None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:01.965365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:01.966119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:01.966446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:01.966750 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:01.967487 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:01.970136 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:02.012981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1366/2754] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:39:01 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:39:02.017333 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:02.030895 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409730b3-6a7d-40cd-b82a-376ecbf18a20 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:02.124217 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:02.125034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:02.125954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:02.126242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:02.127873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:02.130523 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:02.164875 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1389/2755] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:39:02 2022] POST /identity/v3/users => generated 285 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:02.168652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:02.180594 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332883b2-057c-4077-8df9-420cba64109b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:02.186806 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:02.187936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:02.188220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:02.188976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:02.189220 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:02.191397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:02.206017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1367/2756] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:02.209518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:02.222013 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02546702-7608-43cf-a424-10657eb1f58f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:02.230421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162dc83-f86f-4022-acf5-24c33c6f12fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:02.231133 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162dc83-f86f-4022-acf5-24c33c6f12fa admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:02.231546 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162dc83-f86f-4022-acf5-24c33c6f12fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:02.231880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162dc83-f86f-4022-acf5-24c33c6f12fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:02.233698 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162dc83-f86f-4022-acf5-24c33c6f12fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:02.235730 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162dc83-f86f-4022-acf5-24c33c6f12fa admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:02.249050 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1390/2757] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:39:02.252912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:02.264981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca33b5-af00-4d9d-85f9-3ca9a2ae6049 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:02.271356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:02.272269 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:02.272746 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:02.273212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] PATH_INFO: `/v3/projects/1d675b94183348c8b959b132fa70a9d5/users/88086b06f9f646aabd926226365ce6ed/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:02.309872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d675b94183348c8b959b132fa70a9d5, user_id=88086b06f9f646aabd926226365ce6ed, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:02.315244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:02.338123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1368/2758] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:39:02 2022] PUT /identity/v3/projects/1d675b94183348c8b959b132fa70a9d5/users/88086b06f9f646aabd926226365ce6ed/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:02.352071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:02.352071 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:02.352317 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:02.397119 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] MFA Rules not processed for user `88086b06f9f646aabd926226365ce6ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:02.398674 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:02.430993 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:02.477713 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1391/2759] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:02.489422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:02.491189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:02.555369 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d48a0-ec6e-42ef-9904-f8b93a227c53 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:02.559075 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03bc62-b6b1-4fb9-b7e8-5f6dec6f2506 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9be8c8c5f7f6417e9ab19ef5bce0f060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:02.559681 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03bc62-b6b1-4fb9-b7e8-5f6dec6f2506 service glance] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:02.559963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03bc62-b6b1-4fb9-b7e8-5f6dec6f2506 service glance] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:02.560238 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03bc62-b6b1-4fb9-b7e8-5f6dec6f2506 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:02.568657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03bc62-b6b1-4fb9-b7e8-5f6dec6f2506 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:02.570180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03bc62-b6b1-4fb9-b7e8-5f6dec6f2506 service glance] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:02.580240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1369/2760] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:12.474584 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=16 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:12.575700 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:12.705158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:12.706687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:12.838602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f91065-6dbe-4af0-bfaa-82f7c0691df7 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:12.843175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:12.843805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:12.844083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:12.844357 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] PATH_INFO: `/v3/users/414d2a5e741149648268b3d0c1eef021` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:12.845772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] RBAC: Authorizing `identity:delete_user(user_id=414d2a5e741149648268b3d0c1eef021)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:12.847704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:12.894427 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414d2a5e741149648268b3d0c1eef021'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:12.901757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414d2a5e741149648268b3d0c1eef021'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:12.902413 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414d2a5e741149648268b3d0c1eef021'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:12.903053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414d2a5e741149648268b3d0c1eef021'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:12.910100 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414d2a5e741149648268b3d0c1eef021'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:12.927496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1392/2761] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:12 2022] DELETE /identity/v3/users/414d2a5e741149648268b3d0c1eef021 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:12.979550 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7ba79-74e4-4718-821d-032219c2cc26 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:12.979891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7ba79-74e4-4718-821d-032219c2cc26 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:12.980169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7ba79-74e4-4718-821d-032219c2cc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:13.000419 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-9af7ba79-74e4-4718-821d-032219c2cc26 None None] MFA Rules not processed for user `9d593e55f72d4757886261b383e1794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:13.003079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9af7ba79-74e4-4718-821d-032219c2cc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:13.069413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1370/2762] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:13.125613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:13.127750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:13.156717 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:13.208484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e67fe-58e3-469a-bf23-682b5450485a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:13.213254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:13.213919 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:13.214198 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:13.214473 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] PATH_INFO: `/v3/projects/636f8bc0692549f2aa7b000f82149231` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:13.215659 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=636f8bc0692549f2aa7b000f82149231)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:13.217603 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:13.265116 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636f8bc0692549f2aa7b000f82149231', 'request_id': 'req-02c3d59f-41ef-40cd-9c87-4aaee86a0824'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.271064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636f8bc0692549f2aa7b000f82149231', 'request_id': 'req-02c3d59f-41ef-40cd-9c87-4aaee86a0824'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.279649 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636f8bc0692549f2aa7b000f82149231', 'request_id': 'req-02c3d59f-41ef-40cd-9c87-4aaee86a0824'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.291928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] The token cache is being invalidate because project 636f8bc0692549f2aa7b000f82149231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:13.292158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.292799 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.294321 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1393/2763] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:13 2022] DELETE /identity/v3/projects/636f8bc0692549f2aa7b000f82149231 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:13.298947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7ba79-74e4-4718-821d-032219c2cc26 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:13.309481 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7ba79-74e4-4718-821d-032219c2cc26 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:13.381718 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:13.382548 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:13.382870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:13.383221 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] PATH_INFO: `/v3/users/acb79c6d46e547a88a2aefe78b22c0ee` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:13.384381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb79c6d46e547a88a2aefe78b22c0ee)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:13.386506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:13.445122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb79c6d46e547a88a2aefe78b22c0ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.452060 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb79c6d46e547a88a2aefe78b22c0ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.459976 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb79c6d46e547a88a2aefe78b22c0ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.460688 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb79c6d46e547a88a2aefe78b22c0ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.461335 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-557c7d10-e542-4bf0-9ae6-66ddb7abca0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb79c6d46e547a88a2aefe78b22c0ee'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.480380 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1371/2764] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:13 2022] DELETE /identity/v3/users/acb79c6d46e547a88a2aefe78b22c0ee => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:39:13.714024 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:13.714910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:13.732504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3d59f-41ef-40cd-9c87-4aaee86a0824 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:13.799505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:13.800145 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:13.800422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:13.800695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] PATH_INFO: `/v3/projects/6e14f60e46df4243b7d9b78ab67c4856` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:13.801854 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e14f60e46df4243b7d9b78ab67c4856)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:13.804026 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:13.855387 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e14f60e46df4243b7d9b78ab67c4856', 'request_id': 'req-326415cc-1759-423c-a2b5-8f6bca97f3e1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.856602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:13.862821 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e14f60e46df4243b7d9b78ab67c4856', 'request_id': 'req-326415cc-1759-423c-a2b5-8f6bca97f3e1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.871985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e14f60e46df4243b7d9b78ab67c4856', 'request_id': 'req-326415cc-1759-423c-a2b5-8f6bca97f3e1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.887082 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] The token cache is being invalidate because project 6e14f60e46df4243b7d9b78ab67c4856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:13.887567 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.888352 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-326415cc-1759-423c-a2b5-8f6bca97f3e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:13.889972 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1394/2765] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:13 2022] DELETE /identity/v3/projects/6e14f60e46df4243b7d9b78ab67c4856 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:13.895717 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f572783-0242-4664-b587-cfcaead75d1f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:13.896193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f572783-0242-4664-b587-cfcaead75d1f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:13.896298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f572783-0242-4664-b587-cfcaead75d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:13.934579 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:13.965360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5f572783-0242-4664-b587-cfcaead75d1f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:13.966992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5f572783-0242-4664-b587-cfcaead75d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:14.030504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1372/2766] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:14.040102 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d6d02-8ecd-457b-91a8-f583bdaba240 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.040393 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d6d02-8ecd-457b-91a8-f583bdaba240 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.040675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d6d02-8ecd-457b-91a8-f583bdaba240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.064163 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-2f8d6d02-8ecd-457b-91a8-f583bdaba240 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:14.065448 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-2f8d6d02-8ecd-457b-91a8-f583bdaba240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:14.073361 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1395/2767] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:14.077635 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f572783-0242-4664-b587-cfcaead75d1f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.087774 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f572783-0242-4664-b587-cfcaead75d1f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.092981 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.093611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.093918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.094189 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.095399 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:39:14.095970 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.097604 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.114813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1373/2768] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:39:14.116089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d6d02-8ecd-457b-91a8-f583bdaba240 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.125524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d6d02-8ecd-457b-91a8-f583bdaba240 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.130476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.131099 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.131368 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.131640 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.132287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.134223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.167758 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1396/2769] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:39:14 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:39:14.169259 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.181192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d0465-23af-4ea3-8edc-4cc0689d4f2f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.241251 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.241883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.242143 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.242416 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.243048 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.244961 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.275003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1374/2770] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:39:14 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:14.276369 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.287028 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda85591-7c40-4baa-aeb2-cdce21b59f71 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.291880 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.292505 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.292756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.293021 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.293697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.295325 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.308106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1397/2771] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:14.312225 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.324341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7ecac-0239-4e34-8547-d2fd2088bc58 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.328941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80ce34-9d4d-494e-8605-57a6a33b4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.329582 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80ce34-9d4d-494e-8605-57a6a33b4c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.329861 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80ce34-9d4d-494e-8605-57a6a33b4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.330134 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80ce34-9d4d-494e-8605-57a6a33b4c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.330778 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80ce34-9d4d-494e-8605-57a6a33b4c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.332386 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80ce34-9d4d-494e-8605-57a6a33b4c20 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.343302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1375/2772] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:14.348259 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.359510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20ae12-3f4a-4b00-ae0d-12219a64a1b8 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.365885 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.366950 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.367224 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.367493 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] PATH_INFO: `/v3/projects/e100dc10e2094d22a8b81a14002ddecb/users/538a9c25dbc04d0ba5007909b1e1a4ed/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.382500 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e100dc10e2094d22a8b81a14002ddecb, user_id=538a9c25dbc04d0ba5007909b1e1a4ed, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.385139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.399657 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1398/2773] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:39:14 2022] PUT /identity/v3/projects/e100dc10e2094d22a8b81a14002ddecb/users/538a9c25dbc04d0ba5007909b1e1a4ed/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:14.411326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f814b50-d7af-422a-96f4-0e136cd36514 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.411684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f814b50-d7af-422a-96f4-0e136cd36514 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.411999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f814b50-d7af-422a-96f4-0e136cd36514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.441878 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8f814b50-d7af-422a-96f4-0e136cd36514 None None] MFA Rules not processed for user `538a9c25dbc04d0ba5007909b1e1a4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:14.443108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8f814b50-d7af-422a-96f4-0e136cd36514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:14.490852 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:14.509407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1376/2774] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:14.515601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.578171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8b89-04db-4020-9965-b8898872cdbc admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.583295 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.583910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.584194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.584466 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.585090 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.587054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.615279 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1399/2775] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:39:14 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:14.619119 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f814b50-d7af-422a-96f4-0e136cd36514 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.629898 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f814b50-d7af-422a-96f4-0e136cd36514 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.695206 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.695818 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.696095 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.696388 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.697025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.698977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.725978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1377/2776] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:39:14 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:14.727164 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.737871 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dec9d-e83b-4340-aa0a-a9cfa7b62c06 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.743087 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.743941 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.744289 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.744631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.745392 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.747171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.760577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1400/2777] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:14.764705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.776521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f7794-2a71-46cc-a58e-ca53fa5338f0 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.785936 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.787021 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.787843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.788468 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] PATH_INFO: `/v3/projects/6ffad58f801a40d3adbecc5efd4f95a0/users/184f87d175914206aff85dba24705566/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.806198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffad58f801a40d3adbecc5efd4f95a0, user_id=184f87d175914206aff85dba24705566, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.809429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.833102 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1378/2778] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:39:14 2022] PUT /identity/v3/projects/6ffad58f801a40d3adbecc5efd4f95a0/users/184f87d175914206aff85dba24705566/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:39:14.834288 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.861310 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc32463-6d13-4471-8b51-6d0760471826 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.916351 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:14.932621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0845d-5c49-4d8a-a44e-e9d99bedcb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.933560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0845d-5c49-4d8a-a44e-e9d99bedcb4a admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.933940 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0845d-5c49-4d8a-a44e-e9d99bedcb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.934332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0845d-5c49-4d8a-a44e-e9d99bedcb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.935129 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0845d-5c49-4d8a-a44e-e9d99bedcb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.937210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0845d-5c49-4d8a-a44e-e9d99bedcb4a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.950595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1401/2779] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:14.955610 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:14.966440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca7c6-2444-4db2-9f00-9b0652e82a78 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:14.971457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:14.972112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.972392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.972666 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] PATH_INFO: `/v3/projects/6ffad58f801a40d3adbecc5efd4f95a0/users/184f87d175914206aff85dba24705566/roles/2470da926a0342698ca115a00c86fea3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:14.974634 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffad58f801a40d3adbecc5efd4f95a0, user_id=184f87d175914206aff85dba24705566, role_id=2470da926a0342698ca115a00c86fea3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:14.977383 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:14.987831 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1379/2780] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:39:14 2022] PUT /identity/v3/projects/6ffad58f801a40d3adbecc5efd4f95a0/users/184f87d175914206aff85dba24705566/roles/2470da926a0342698ca115a00c86fea3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:39:14.997996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:14.998417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:14.998888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:15.028938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] MFA Rules not processed for user `184f87d175914206aff85dba24705566`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:15.030173 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:15.103958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1402/2781] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:15.115312 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:15.117575 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:15.191789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23804761-a9c4-41d5-8008-e2d958a547ee admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:15.195464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:15.196065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:15.196345 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:15.196619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:15.243761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:15.260535 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:15.263760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:15.275276 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1380/2782] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:19.514727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:19.516617 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:19.529061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc1229-3ec0-4ebf-88b9-b81446ec4ea5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:19.533424 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:19.534034 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:19.534313 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:19.534582 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:19.543728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:19.545158 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:19.555066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1403/2783] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:24.928608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:24.930104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:25.021108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863d1a9-7649-4a01-b255-7fc42398e516 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:25.026285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eecf9-eed7-4313-a83f-9cb5012fe2a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:25.026885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eecf9-eed7-4313-a83f-9cb5012fe2a3 service placement] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:25.027168 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eecf9-eed7-4313-a83f-9cb5012fe2a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:25.027441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eecf9-eed7-4313-a83f-9cb5012fe2a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:25.029148 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-268eecf9-eed7-4313-a83f-9cb5012fe2a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:25.040921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eecf9-eed7-4313-a83f-9cb5012fe2a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:25.042545 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eecf9-eed7-4313-a83f-9cb5012fe2a3 service placement] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:25.052326 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1381/2784] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:27.673382 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:27.740265 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1ff40-3a2a-4490-bf95-4c07de7e3e31 service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:27.745051 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:27.745712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:27.745986 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:27.746277 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] PATH_INFO: `/v3/users/538a9c25dbc04d0ba5007909b1e1a4ed` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:27.747377 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=538a9c25dbc04d0ba5007909b1e1a4ed)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:27.749297 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:27.798029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538a9c25dbc04d0ba5007909b1e1a4ed'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:27.806492 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538a9c25dbc04d0ba5007909b1e1a4ed'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:27.807286 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538a9c25dbc04d0ba5007909b1e1a4ed'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:27.807979 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538a9c25dbc04d0ba5007909b1e1a4ed'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:27.812749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:27.817120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538a9c25dbc04d0ba5007909b1e1a4ed'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:27.839020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1404/2785] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:27 2022] DELETE /identity/v3/users/538a9c25dbc04d0ba5007909b1e1a4ed => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:27.843569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:27.844056 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:27.844165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:27.868533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:27.869849 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:27.936626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1382/2786] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:27.943348 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:27.944725 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:28.011010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66dae2-b37a-46ae-a4c9-3380009d5831 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:28.015023 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:28.015634 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:28.015913 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:28.016184 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:28.025802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:28.027292 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:28.038841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1405/2787] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:28.304372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:28.305938 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:28.320903 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c05e2-bec0-49fa-bc5a-95edd079ed64 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:28.327413 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:28.329025 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:28.329682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:28.330091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] PATH_INFO: `/v3/projects/e100dc10e2094d22a8b81a14002ddecb` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:28.331984 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e100dc10e2094d22a8b81a14002ddecb)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:28.333943 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:28.380727 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:28.384013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e100dc10e2094d22a8b81a14002ddecb', 'request_id': 'req-f31b51dc-efc4-4daa-9e48-aca5757e6833'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.390027 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e100dc10e2094d22a8b81a14002ddecb', 'request_id': 'req-f31b51dc-efc4-4daa-9e48-aca5757e6833'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.397476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e100dc10e2094d22a8b81a14002ddecb', 'request_id': 'req-f31b51dc-efc4-4daa-9e48-aca5757e6833'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.410827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] The token cache is being invalidate because project e100dc10e2094d22a8b81a14002ddecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:28.410827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.410827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.411877 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1383/2788] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:28 2022] DELETE /identity/v3/projects/e100dc10e2094d22a8b81a14002ddecb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:28.415990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:28.425739 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244163-59b8-4833-83d9-5814dd487b35 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:28.483324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:28.483997 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:28.484474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:28.484922 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] PATH_INFO: `/v3/users/184f87d175914206aff85dba24705566` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:28.486843 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=184f87d175914206aff85dba24705566)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:28.491127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:28.536761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184f87d175914206aff85dba24705566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.544114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184f87d175914206aff85dba24705566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.544773 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184f87d175914206aff85dba24705566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.545417 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184f87d175914206aff85dba24705566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.552996 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-58984bff-ee7c-44d2-a734-e37ad8de5d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184f87d175914206aff85dba24705566'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:28.566574 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:28.572738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1406/2789] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:28 2022] DELETE /identity/v3/users/184f87d175914206aff85dba24705566 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:28.950977 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:28.952472 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:28.966867 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b51dc-efc4-4daa-9e48-aca5757e6833 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.037763 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.038397 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.038682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.038958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] PATH_INFO: `/v3/projects/6ffad58f801a40d3adbecc5efd4f95a0` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.040085 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffad58f801a40d3adbecc5efd4f95a0)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.042036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.090475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffad58f801a40d3adbecc5efd4f95a0', 'request_id': 'req-7838d67c-c97c-479c-badc-3d6e25d2f32e'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:29.097182 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffad58f801a40d3adbecc5efd4f95a0', 'request_id': 'req-7838d67c-c97c-479c-badc-3d6e25d2f32e'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:29.107264 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffad58f801a40d3adbecc5efd4f95a0', 'request_id': 'req-7838d67c-c97c-479c-badc-3d6e25d2f32e'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:29.120656 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] The token cache is being invalidate because project 6ffad58f801a40d3adbecc5efd4f95a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:29.120927 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:29.122103 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-7838d67c-c97c-479c-badc-3d6e25d2f32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:29.124186 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1384/2790] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:28 2022] DELETE /identity/v3/projects/6ffad58f801a40d3adbecc5efd4f95a0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:29.130131 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6ed7-5a37-450e-9017-7735e799af02 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.130641 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6ed7-5a37-450e-9017-7735e799af02 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.130741 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6ed7-5a37-450e-9017-7735e799af02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.162479 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-04ce6ed7-5a37-450e-9017-7735e799af02 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:29.163728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-04ce6ed7-5a37-450e-9017-7735e799af02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:29.242488 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1407/2791] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:29.251701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc45b-18f0-4c72-a091-c4e09c6d391c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.252039 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc45b-18f0-4c72-a091-c4e09c6d391c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.252260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc45b-18f0-4c72-a091-c4e09c6d391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.283172 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8a2fc45b-18f0-4c72-a091-c4e09c6d391c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:29.284498 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8a2fc45b-18f0-4c72-a091-c4e09c6d391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:29.294569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1385/2792] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:29.299225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce6ed7-5a37-450e-9017-7735e799af02 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.309566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce6ed7-5a37-450e-9017-7735e799af02 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.314759 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.316159 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.316510 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.316781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.317457 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:39:29.319142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.319346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.333114 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1408/2793] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:39:29.337512 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fc45b-18f0-4c72-a091-c4e09c6d391c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.350407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fc45b-18f0-4c72-a091-c4e09c6d391c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.363689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.364321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.364622 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.364868 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.365559 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.367476 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.405235 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1386/2794] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:39:29 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:29.406598 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.418745 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a55f5d-ee1c-4e72-9682-e46640e25d45 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.491733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.492345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.492625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.492915 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.493561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.495476 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.521200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1409/2795] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:39:29 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:29.522306 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.532598 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97c63-ca75-44da-8b5b-ac318275f976 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.537716 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.538378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.538655 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.538922 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.539556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.541201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.552963 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1387/2796] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:29.556933 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.567366 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7354063-4816-422b-835b-714d4f17157d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.572254 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fc40a-a6ce-4d54-9472-84110009a469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.572796 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc40a-a6ce-4d54-9472-84110009a469 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.573074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc40a-a6ce-4d54-9472-84110009a469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.573372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc40a-a6ce-4d54-9472-84110009a469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.574022 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fc40a-a6ce-4d54-9472-84110009a469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.575707 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fc40a-a6ce-4d54-9472-84110009a469 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.590020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1410/2797] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:29.594069 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.605077 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d9922-08c1-4ec0-916b-edd974fdef42 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.610059 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.611442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.611442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.611442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] PATH_INFO: `/v3/projects/160c761e57b1475a8d22129cf7e19be3/users/4a92daa4533542d98bf2e218cb20c2fb/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.632830 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=160c761e57b1475a8d22129cf7e19be3, user_id=4a92daa4533542d98bf2e218cb20c2fb, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.635776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.646122 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:29.653111 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1388/2798] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:39:29 2022] PUT /identity/v3/projects/160c761e57b1475a8d22129cf7e19be3/users/4a92daa4533542d98bf2e218cb20c2fb/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:39:29.664148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040e84a-7eda-43fc-81c7-ab2b6a9c25a2 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.664454 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040e84a-7eda-43fc-81c7-ab2b6a9c25a2 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.664724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040e84a-7eda-43fc-81c7-ab2b6a9c25a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.696324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-e040e84a-7eda-43fc-81c7-ab2b6a9c25a2 None None] MFA Rules not processed for user `4a92daa4533542d98bf2e218cb20c2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:29.697720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e040e84a-7eda-43fc-81c7-ab2b6a9c25a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:29.763865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:29.776637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1411/2799] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:29.788581 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.790365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:29.857576 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904aaf-5f7d-4a8d-addd-5ab3b9a886ee admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.861442 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3af98-7c32-480c-a688-78e326927c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.862100 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3af98-7c32-480c-a688-78e326927c44 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.862385 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3af98-7c32-480c-a688-78e326927c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.862657 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3af98-7c32-480c-a688-78e326927c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.872046 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3af98-7c32-480c-a688-78e326927c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.873556 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3af98-7c32-480c-a688-78e326927c44 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:29.887321 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1389/2800] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:29.908802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040e84a-7eda-43fc-81c7-ab2b6a9c25a2 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:29.969180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040e84a-7eda-43fc-81c7-ab2b6a9c25a2 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:29.974227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:29.974848 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:29.975120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:29.975388 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] PATH_INFO: `/v3/users/4a92daa4533542d98bf2e218cb20c2fb` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:29.976511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a92daa4533542d98bf2e218cb20c2fb)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:29.978455 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.022687 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a92daa4533542d98bf2e218cb20c2fb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.029490 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a92daa4533542d98bf2e218cb20c2fb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.030106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a92daa4533542d98bf2e218cb20c2fb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.030705 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a92daa4533542d98bf2e218cb20c2fb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.037443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a92daa4533542d98bf2e218cb20c2fb'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.053957 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1412/2801] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:29 2022] DELETE /identity/v3/users/4a92daa4533542d98bf2e218cb20c2fb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:30.058695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.059150 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.059244 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.078709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:30.079912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:30.138404 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1390/2802] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:30.147805 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:30.149180 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:30.224947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6f154-ff87-4250-9f8c-3c18148acdb8 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:30.228556 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74fbb4-9192-4cac-9473-c413a12ff725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:30.229148 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74fbb4-9192-4cac-9473-c413a12ff725 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.229456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74fbb4-9192-4cac-9473-c413a12ff725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.229728 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74fbb4-9192-4cac-9473-c413a12ff725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.236504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:30.239343 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74fbb4-9192-4cac-9473-c413a12ff725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:30.241230 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74fbb4-9192-4cac-9473-c413a12ff725 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.251978 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1413/2803] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:30.516531 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:30.517965 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:30.529797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adcbad-d605-444f-97f4-5bf82013da7c None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:30.534921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:30.535555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.535837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.536114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] PATH_INFO: `/v3/projects/160c761e57b1475a8d22129cf7e19be3` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.537229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] RBAC: Authorizing `identity:delete_project(project_id=160c761e57b1475a8d22129cf7e19be3)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:30.539171 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.589521 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160c761e57b1475a8d22129cf7e19be3', 'request_id': 'req-99273854-3a42-456d-a1bd-9e17c828810a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.595310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160c761e57b1475a8d22129cf7e19be3', 'request_id': 'req-99273854-3a42-456d-a1bd-9e17c828810a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.603745 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160c761e57b1475a8d22129cf7e19be3', 'request_id': 'req-99273854-3a42-456d-a1bd-9e17c828810a'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.615542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] The token cache is being invalidate because project 160c761e57b1475a8d22129cf7e19be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:30.615828 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.616441 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-99273854-3a42-456d-a1bd-9e17c828810a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:30.617968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1391/2804] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:30 2022] DELETE /identity/v3/projects/160c761e57b1475a8d22129cf7e19be3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:30.622847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770dae6-1cbd-4b5b-8ae2-68b4fd5fa51a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.623132 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770dae6-1cbd-4b5b-8ae2-68b4fd5fa51a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.623405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770dae6-1cbd-4b5b-8ae2-68b4fd5fa51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.642958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-8770dae6-1cbd-4b5b-8ae2-68b4fd5fa51a None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:30.644267 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-8770dae6-1cbd-4b5b-8ae2-68b4fd5fa51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:30.700566 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1414/2805] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:30.709392 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65bbcf-a52a-40b2-bd88-96866cd346e4 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.709699 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65bbcf-a52a-40b2-bd88-96866cd346e4 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.709972 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65bbcf-a52a-40b2-bd88-96866cd346e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.729748 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-3e65bbcf-a52a-40b2-bd88-96866cd346e4 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:30.730941 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-3e65bbcf-a52a-40b2-bd88-96866cd346e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:30.739274 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1392/2806] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:30.743446 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770dae6-1cbd-4b5b-8ae2-68b4fd5fa51a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:30.753176 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770dae6-1cbd-4b5b-8ae2-68b4fd5fa51a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:30.757651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:30.758236 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.758511 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.758776 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.759420 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:39:30.759716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:30.761318 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.772422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1415/2807] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:39:30.776379 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65bbcf-a52a-40b2-bd88-96866cd346e4 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:30.785866 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65bbcf-a52a-40b2-bd88-96866cd346e4 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:30.790420 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:30.790999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.791271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.791537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.792151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:30.794105 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.814467 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:30.823328 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1393/2808] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:39:30 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:30.827229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:30.838015 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efe017-e4a3-42fe-9894-0d5133ce59f9 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:30.892683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:30.893748 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.894030 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.894302 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.894932 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:30.896845 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.923841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1416/2809] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:39:30 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:39:30.925809 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:30.936155 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40d26c-d07f-4b54-aee1-fa5a971f6ce4 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:30.940795 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:30.941494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.941840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.942228 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.942986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:30.944632 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.954537 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1394/2810] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:30.959210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:30.969637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035141d-66fa-4102-a889-97312801c24a admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:30.975736 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b7e50-342f-4a09-98ac-bf9df0de6bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:30.976421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7e50-342f-4a09-98ac-bf9df0de6bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:30.976904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7e50-342f-4a09-98ac-bf9df0de6bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:30.977202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7e50-342f-4a09-98ac-bf9df0de6bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:30.977873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b7e50-342f-4a09-98ac-bf9df0de6bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:30.979474 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b7e50-342f-4a09-98ac-bf9df0de6bd1 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:30.990301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1417/2811] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:30.994698 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:31.004885 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745be32-6b19-46fa-80fc-5255627adcc9 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:31.009542 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:31.010192 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:31.010469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:31.010737 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] PATH_INFO: `/v3/projects/a359e288e760471c8258ab612c6baf13/users/0511fcefa4bb43548dd95a84af4d7d88/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:31.025998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a359e288e760471c8258ab612c6baf13, user_id=0511fcefa4bb43548dd95a84af4d7d88, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:31.028630 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:31.041611 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1395/2812] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:39:30 2022] PUT /identity/v3/projects/a359e288e760471c8258ab612c6baf13/users/0511fcefa4bb43548dd95a84af4d7d88/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:31.051928 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd182c6-c6d1-4d66-8c2d-1b0a66dd6025 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:31.052239 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd182c6-c6d1-4d66-8c2d-1b0a66dd6025 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:31.052512 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd182c6-c6d1-4d66-8c2d-1b0a66dd6025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:31.079817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-4fd182c6-c6d1-4d66-8c2d-1b0a66dd6025 None None] MFA Rules not processed for user `0511fcefa4bb43548dd95a84af4d7d88`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:31.081496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-4fd182c6-c6d1-4d66-8c2d-1b0a66dd6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:31.128930 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:31.146594 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1418/2813] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:31.157686 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:31.159956 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:31.214846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a69ca2-80dd-4be5-a442-5e567f6c18e8 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:31.218410 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:31.218996 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:31.219271 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:31.219539 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:31.228240 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:31.229695 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:31.238791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1396/2814] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:41.143194 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=4 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:41.234314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:44.640824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4116215-c3af-403b-b17d-fd6fbb969453 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:44.641443 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4116215-c3af-403b-b17d-fd6fbb969453 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:44.641613 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4116215-c3af-403b-b17d-fd6fbb969453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:44.670651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-a4116215-c3af-403b-b17d-fd6fbb969453 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:44.671849 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-a4116215-c3af-403b-b17d-fd6fbb969453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:44.734002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1419/2815] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 10 12:39:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:44.778065 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:44.779456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:44.850031 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aee43f-a788-4751-ae38-6b2468660159 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:44.855088 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:44.855637 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:44.855915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:44.856187 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:44.864658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:44.866283 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:44.874258 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1397/2816] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:39:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:39:45.331097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce314f87-d5e2-4961-94ab-718302b51293 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:45.331432 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce314f87-d5e2-4961-94ab-718302b51293 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:45.331708 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce314f87-d5e2-4961-94ab-718302b51293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:45.351526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-ce314f87-d5e2-4961-94ab-718302b51293 None None] MFA Rules not processed for user `770127af5f494b7e9198438441868fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:45.353456 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ce314f87-d5e2-4961-94ab-718302b51293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:45.411498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1420/2817] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:46.823728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:46.827032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:46.971930 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8a404-605c-41be-ac34-cac111355c56 service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:46.980522 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:46.981554 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:46.982079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:46.982591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] PATH_INFO: `/v3/users/88086b06f9f646aabd926226365ce6ed` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:46.984302 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88086b06f9f646aabd926226365ce6ed)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:46.986915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:47.099687 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88086b06f9f646aabd926226365ce6ed'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.110802 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88086b06f9f646aabd926226365ce6ed'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.124350 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88086b06f9f646aabd926226365ce6ed'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.125218 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88086b06f9f646aabd926226365ce6ed'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.126273 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88086b06f9f646aabd926226365ce6ed'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.154057 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1398/2818] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:46 2022] DELETE /identity/v3/users/88086b06f9f646aabd926226365ce6ed => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:47.158043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:47.159139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:47.159637 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:47.182384 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:47.183927 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:47.250210 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1421/2819] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:47.260690 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:47.262402 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:47.274761 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:47.324808 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43a567-388d-4157-82d8-b90a1d6d0804 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:47.328540 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfce85-3ba6-4208-bbfc-3348dfa2c603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:47.329151 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfce85-3ba6-4208-bbfc-3348dfa2c603 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:47.329463 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfce85-3ba6-4208-bbfc-3348dfa2c603 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:47.329738 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfce85-3ba6-4208-bbfc-3348dfa2c603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:47.339198 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfce85-3ba6-4208-bbfc-3348dfa2c603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:47.340646 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfce85-3ba6-4208-bbfc-3348dfa2c603 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:47.351591 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1399/2820] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:47.732983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:47.734706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:47.756468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a8620-958b-418f-80e6-e69c492eba98 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:47.763066 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:47.763824 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:47.764197 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:47.764565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] PATH_INFO: `/v3/projects/1d675b94183348c8b959b132fa70a9d5` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:47.767053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d675b94183348c8b959b132fa70a9d5)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:47.768884 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:47.825205 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:47.864515 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d675b94183348c8b959b132fa70a9d5', 'request_id': 'req-f1368f55-eda6-490d-bd8d-ebbbd8935c96'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.877560 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d675b94183348c8b959b132fa70a9d5', 'request_id': 'req-f1368f55-eda6-490d-bd8d-ebbbd8935c96'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.888656 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d675b94183348c8b959b132fa70a9d5', 'request_id': 'req-f1368f55-eda6-490d-bd8d-ebbbd8935c96'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.904686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] The token cache is being invalidate because project 1d675b94183348c8b959b132fa70a9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:47.905146 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.906287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f1368f55-eda6-490d-bd8d-ebbbd8935c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:47.908107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1422/2821] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:47 2022] DELETE /identity/v3/projects/1d675b94183348c8b959b132fa70a9d5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:39:47.913565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ab1f-bf61-43ac-93be-5c429e42748f None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:47.914475 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ab1f-bf61-43ac-93be-5c429e42748f None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:47.914923 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ab1f-bf61-43ac-93be-5c429e42748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:47.935837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e113ab1f-bf61-43ac-93be-5c429e42748f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:47.937079 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e113ab1f-bf61-43ac-93be-5c429e42748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:48.022114 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1400/2822] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:48.024110 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e4163-41b8-4022-9fac-11e90372f1e3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.024531 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e4163-41b8-4022-9fac-11e90372f1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.024909 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e4163-41b8-4022-9fac-11e90372f1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.055646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9e3e4163-41b8-4022-9fac-11e90372f1e3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:48.057196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9e3e4163-41b8-4022-9fac-11e90372f1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:48.069314 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1423/2823] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:48.070912 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113ab1f-bf61-43ac-93be-5c429e42748f None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:48.086300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113ab1f-bf61-43ac-93be-5c429e42748f None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:48.100257 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:48.101348 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.101658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.102178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.103022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:39:48.103840 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:48.105007 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:48.123871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1401/2824] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 12:39:48.128175 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e4163-41b8-4022-9fac-11e90372f1e3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:48.138936 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e4163-41b8-4022-9fac-11e90372f1e3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:48.149478 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:48.150156 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.150438 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.150712 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.151383 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:48.153422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:48.193353 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1424/2825] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:39:48 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:39:48.197415 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:48.209856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77775e09-c035-4188-a6e6-8ba601faff0a admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:48.355484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:48.356343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.356790 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.357201 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.357987 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:48.361154 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:48.488041 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:48.489967 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1402/2826] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:39:48 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:39:48.525097 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b726285-63f4-4d7c-82c9-8ac9166487c6 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:48.531820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:48.532624 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.532904 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.533171 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.534095 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:48.536840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:48.550638 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1425/2827] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:48.555891 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:48.573446 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a8d81-546b-49f8-b3ae-a75ced114043 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:48.589491 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dcd4c-9aeb-4aa8-8538-c8a45d6ec8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:48.590147 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dcd4c-9aeb-4aa8-8538-c8a45d6ec8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.590429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dcd4c-9aeb-4aa8-8538-c8a45d6ec8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.591526 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dcd4c-9aeb-4aa8-8538-c8a45d6ec8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.592108 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dcd4c-9aeb-4aa8-8538-c8a45d6ec8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:48.599899 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dcd4c-9aeb-4aa8-8538-c8a45d6ec8f2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:48.642429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1403/2828] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:39:48.643475 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:48.682924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6df45-ba05-4cc1-a483-d6104e1aa9dd admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:48.713826 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:48.724141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.724737 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.725636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] PATH_INFO: `/v3/projects/f797ec18d0a847748116b1dd18bc8cf1/users/41879b8ab41b49f79ad7f8ae4871f251/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.762691 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f797ec18d0a847748116b1dd18bc8cf1, user_id=41879b8ab41b49f79ad7f8ae4871f251, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:48.766042 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:48.799990 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1426/2829] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:39:48 2022] PUT /identity/v3/projects/f797ec18d0a847748116b1dd18bc8cf1/users/41879b8ab41b49f79ad7f8ae4871f251/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:48.806364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef3f5f-241e-498c-b063-bd2b9bc6857c None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:48.806570 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef3f5f-241e-498c-b063-bd2b9bc6857c None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:48.807360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef3f5f-241e-498c-b063-bd2b9bc6857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:48.836061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-66ef3f5f-241e-498c-b063-bd2b9bc6857c None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:48.838237 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-66ef3f5f-241e-498c-b063-bd2b9bc6857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:48.852489 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:48.993061 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1404/2830] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:49.000200 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:49.002005 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:49.068127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72865ded-5f70-48d5-abd2-41636ffab4ab admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:49.071810 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:49.072652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:49.072954 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:49.073233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:49.085678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:49.087595 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:49.099850 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1427/2831] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:39:49.852856 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766a788-9715-4422-8af4-2f5fd8c11e19 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:49.853120 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766a788-9715-4422-8af4-2f5fd8c11e19 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:49.853511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766a788-9715-4422-8af4-2f5fd8c11e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:49.893638 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-e766a788-9715-4422-8af4-2f5fd8c11e19 None None] MFA Rules not processed for user `41879b8ab41b49f79ad7f8ae4871f251`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:49.894902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e766a788-9715-4422-8af4-2f5fd8c11e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:49.981365 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1405/2832] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:39:49.991174 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:49.992788 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:50.080437 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:50.112803 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a99109-35b4-46a3-92a9-200dc35e6688 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:50.118328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:50.119073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:50.119480 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:50.119846 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:50.139733 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:50.142506 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:50.159421 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1428/2833] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:52.733946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005695d-9f62-444f-a11d-433faacf370e None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:52.734484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005695d-9f62-444f-a11d-433faacf370e None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:52.734484 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005695d-9f62-444f-a11d-433faacf370e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:52.767112 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-5005695d-9f62-444f-a11d-433faacf370e None None] MFA Rules not processed for user `41879b8ab41b49f79ad7f8ae4871f251`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:52.770167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-5005695d-9f62-444f-a11d-433faacf370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:52.785184 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1406/2834] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:56.927809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:56.929521 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:56.944123 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e674-b5f2-424c-88fa-00591b79cb2a service cinder] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:56.949010 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:56.949761 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:56.950242 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:56.950692 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] PATH_INFO: `/v3/users/0511fcefa4bb43548dd95a84af4d7d88` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:56.952061 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0511fcefa4bb43548dd95a84af4d7d88)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:56.954054 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:57.008462 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0511fcefa4bb43548dd95a84af4d7d88'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.017724 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0511fcefa4bb43548dd95a84af4d7d88'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.018408 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0511fcefa4bb43548dd95a84af4d7d88'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.018992 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0511fcefa4bb43548dd95a84af4d7d88'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.027484 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0511fcefa4bb43548dd95a84af4d7d88'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.074800 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1429/2835] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:39:56 2022] DELETE /identity/v3/users/0511fcefa4bb43548dd95a84af4d7d88 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:39:57.078042 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:57.078525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:57.078843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:57.108742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:39:57.110900 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:57.223902 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1407/2836] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:39:57.235183 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:57.237485 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:57.306599 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203b9c7-f974-4e01-b821-1ed11cd8cc04 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:57.312440 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c81a9-d317-4e23-b8c9-4f23b896487b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:57.313334 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c81a9-d317-4e23-b8c9-4f23b896487b service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:57.313720 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c81a9-d317-4e23-b8c9-4f23b896487b service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:57.314153 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c81a9-d317-4e23-b8c9-4f23b896487b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:57.328924 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c81a9-d317-4e23-b8c9-4f23b896487b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:57.332496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c81a9-d317-4e23-b8c9-4f23b896487b service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:57.348771 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1430/2837] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:39:57.755492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:39:57.757055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:39:57.773008 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f5843-87b8-498a-9bc1-185fb6c5ea51 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:39:57.778879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:39:57.779767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:39:57.780440 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:39:57.781036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] PATH_INFO: `/v3/projects/a359e288e760471c8258ab612c6baf13` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:39:57.782432 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a359e288e760471c8258ab612c6baf13)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:39:57.784360 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:39:57.831571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:39:57.843728 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a359e288e760471c8258ab612c6baf13', 'request_id': 'req-934f7812-e5b8-47df-bcb6-7e2370950487'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.852013 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a359e288e760471c8258ab612c6baf13', 'request_id': 'req-934f7812-e5b8-47df-bcb6-7e2370950487'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.861097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a359e288e760471c8258ab612c6baf13', 'request_id': 'req-934f7812-e5b8-47df-bcb6-7e2370950487'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.873459 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] The token cache is being invalidate because project a359e288e760471c8258ab612c6baf13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:39:57.873760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.874492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:39:57.875998 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1408/2838] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:39:57 2022] DELETE /identity/v3/projects/a359e288e760471c8258ab612c6baf13 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:40:04.992989 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:04.993460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:04.993554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:05.012578 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] MFA Rules not processed for user `9c8a6dd668474aec9274ca29756be19d`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:40:05.013795 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:05.059544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:05.069008 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1431/2839] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 10 12:40:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:40:05.080894 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:05.082920 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:05.149742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f7812-e5b8-47df-bcb6-7e2370950487 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:05.154620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb6a74548184806972a7f8deb30f789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:05.155209 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:05.155485 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:05.155755 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:05.163494 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:05.165054 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:05.172879 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1409/2840] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 10 12:40:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 12:40:07.418798 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:07.424482 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:07.500625 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f075f1-c229-4b34-9dcf-2f4a9db878c5 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:07.505863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:07.506575 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:07.507346 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:07.507618 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] PATH_INFO: `/v3/users/9d593e55f72d4757886261b383e1794f` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:07.508825 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d593e55f72d4757886261b383e1794f)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:07.510809 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:07.558636 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d593e55f72d4757886261b383e1794f'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.567003 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d593e55f72d4757886261b383e1794f'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.567621 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d593e55f72d4757886261b383e1794f'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.568227 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d593e55f72d4757886261b383e1794f'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.577646 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-f05af664-6d38-475a-9e6b-d8a32d923897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d593e55f72d4757886261b383e1794f'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.596716 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1432/2841] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:40:07 2022] DELETE /identity/v3/users/9d593e55f72d4757886261b383e1794f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:40:07.781300 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:07.793568 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0522ef-e340-481b-a618-f2e9c1311368 service swift] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:07.862658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:07.863398 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:07.863684 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:07.863958 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] PATH_INFO: `/v3/projects/afa4735b7c5b46bc9a0ca81f4733de00` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:07.865101 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa4735b7c5b46bc9a0ca81f4733de00)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:07.867609 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:07.918308 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa4735b7c5b46bc9a0ca81f4733de00', 'request_id': 'req-928c6708-297b-4913-99bb-43f7ac91d0da'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.925178 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:07.926701 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa4735b7c5b46bc9a0ca81f4733de00', 'request_id': 'req-928c6708-297b-4913-99bb-43f7ac91d0da'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.937658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa4735b7c5b46bc9a0ca81f4733de00', 'request_id': 'req-928c6708-297b-4913-99bb-43f7ac91d0da'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.949288 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] The token cache is being invalidate because project afa4735b7c5b46bc9a0ca81f4733de00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:40:07.949767 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.950431 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-928c6708-297b-4913-99bb-43f7ac91d0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:07.954787 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1410/2842] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:40:07 2022] DELETE /identity/v3/projects/afa4735b7c5b46bc9a0ca81f4733de00 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:40:07.957142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133741a-e4bc-4583-b472-2eeb06e2c6f3 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:07.957472 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133741a-e4bc-4583-b472-2eeb06e2c6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:07.957749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133741a-e4bc-4583-b472-2eeb06e2c6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:07.977862 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-9133741a-e4bc-4583-b472-2eeb06e2c6f3 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:40:07.979077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-9133741a-e4bc-4583-b472-2eeb06e2c6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:08.040012 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1433/2843] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:40:08.047613 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b54b3-22dc-4aa9-a5c5-27f302089b58 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.047966 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b54b3-22dc-4aa9-a5c5-27f302089b58 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.048303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b54b3-22dc-4aa9-a5c5-27f302089b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.075280 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-8f0b54b3-22dc-4aa9-a5c5-27f302089b58 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:40:08.076592 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8f0b54b3-22dc-4aa9-a5c5-27f302089b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:08.084488 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1411/2844] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:40:08.088682 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133741a-e4bc-4583-b472-2eeb06e2c6f3 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.100731 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133741a-e4bc-4583-b472-2eeb06e2c6f3 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.105920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.106554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.106829 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.107098 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.107750 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80063) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:40:08.108385 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.109697 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.122269 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1434/2845] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:40:08.126278 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b54b3-22dc-4aa9-a5c5-27f302089b58 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.138955 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b54b3-22dc-4aa9-a5c5-27f302089b58 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.145704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.146341 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.146682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.146959 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] PATH_INFO: `/v3/projects` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.147645 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.149652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.182371 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1412/2846] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:40:08 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:40:08.186518 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.197139 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca953-3b60-4d2b-9d70-32a99bd5b1fe admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.201324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:08.276704 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.277401 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.277688 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.277960 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] PATH_INFO: `/v3/users` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.278601 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.280559 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.307863 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1435/2847] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:40:08 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 12:40:08.312026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.323552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14cb96-641f-4498-ad83-ec28c31d507c admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.328918 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.329789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.330072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.330340 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.330990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.332603 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.350915 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1413/2848] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 12:40:08.352544 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.364177 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0a7a0-44a1-4254-969b-3f63b0c3bbdf admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.375869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b5c77-7ba6-44c2-91c9-ff2debf3e8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.376503 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b5c77-7ba6-44c2-91c9-ff2debf3e8dd admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.376818 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b5c77-7ba6-44c2-91c9-ff2debf3e8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.377096 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b5c77-7ba6-44c2-91c9-ff2debf3e8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.377792 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b5c77-7ba6-44c2-91c9-ff2debf3e8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.379422 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b5c77-7ba6-44c2-91c9-ff2debf3e8dd admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.392561 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1436/2849] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:40:08.396883 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.410629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750bd2a-668a-454a-83c6-81eb320efbd2 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.418680 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.420135 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] REQUEST_METHOD: `PUT` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.420650 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.421133 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] PATH_INFO: `/v3/projects/9db06e0049614700a09a866ed046d68c/users/aa4f66eef63a446490ffeeff28e5cd60/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.438313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db06e0049614700a09a866ed046d68c, user_id=aa4f66eef63a446490ffeeff28e5cd60, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.441207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.461897 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1414/2850] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:40:08 2022] PUT /identity/v3/projects/9db06e0049614700a09a866ed046d68c/users/aa4f66eef63a446490ffeeff28e5cd60/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:40:08.473706 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c6a4-03b7-4e7b-8ad5-9248e5a9d51a None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.474639 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c6a4-03b7-4e7b-8ad5-9248e5a9d51a None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.474912 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c6a4-03b7-4e7b-8ad5-9248e5a9d51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.504101 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-29f8c6a4-03b7-4e7b-8ad5-9248e5a9d51a None None] MFA Rules not processed for user `aa4f66eef63a446490ffeeff28e5cd60`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:40:08.505336 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-29f8c6a4-03b7-4e7b-8ad5-9248e5a9d51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:08.570651 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1437/2851] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:40:08.581830 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.583533 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:08.649097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862577-61f8-450d-b36f-4c619aea9932 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.652789 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd10f77-b272-4d85-b78e-795872c4a81b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.653428 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd10f77-b272-4d85-b78e-795872c4a81b service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.653705 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd10f77-b272-4d85-b78e-795872c4a81b service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.653975 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd10f77-b272-4d85-b78e-795872c4a81b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.667026 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd10f77-b272-4d85-b78e-795872c4a81b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.668596 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd10f77-b272-4d85-b78e-795872c4a81b service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.678094 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1415/2852] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:40:08.704278 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c6a4-03b7-4e7b-8ad5-9248e5a9d51a None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.768554 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c6a4-03b7-4e7b-8ad5-9248e5a9d51a None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:08.773820 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:08.774527 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.774802 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.775074 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] PATH_INFO: `/v3/users/aa4f66eef63a446490ffeeff28e5cd60` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.776379 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4f66eef63a446490ffeeff28e5cd60)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:08.778355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:08.797376 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:08.836835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4f66eef63a446490ffeeff28e5cd60'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:08.848607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4f66eef63a446490ffeeff28e5cd60'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:08.849541 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4f66eef63a446490ffeeff28e5cd60'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:08.850141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4f66eef63a446490ffeeff28e5cd60'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:08.857807 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4f66eef63a446490ffeeff28e5cd60'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:08.878111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1438/2853] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:40:08 2022] DELETE /identity/v3/users/aa4f66eef63a446490ffeeff28e5cd60 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:40:08.879855 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:08.880364 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:08.880628 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:08.901901 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:40:08.903284 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:08.968942 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1416/2854] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:40:08.979106 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:08.980602 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:09.044282 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff332c4-a052-4f9a-93a4-4d4db0f5cf7f admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:09.048322 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bb987-ad4a-4572-a27b-2e330a5b480e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:09.049124 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bb987-ad4a-4572-a27b-2e330a5b480e service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:09.049565 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bb987-ad4a-4572-a27b-2e330a5b480e service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:09.049939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bb987-ad4a-4572-a27b-2e330a5b480e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:09.063558 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bb987-ad4a-4572-a27b-2e330a5b480e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:09.065031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bb987-ad4a-4572-a27b-2e330a5b480e service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:09.075328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1439/2855] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:40:09.374504 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:40:09.375985 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:09.392310 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca34c-bda6-44fb-902b-3b2d23164ff6 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:40:09.398871 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:40:09.399776 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:09.400179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:09.400555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] PATH_INFO: `/v3/projects/9db06e0049614700a09a866ed046d68c` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:09.401858 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db06e0049614700a09a866ed046d68c)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:40:09.404167 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:40:09.434301 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:09.462525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db06e0049614700a09a866ed046d68c', 'request_id': 'req-b13ef066-d4c9-4695-8120-901c32e2802f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:09.468706 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db06e0049614700a09a866ed046d68c', 'request_id': 'req-b13ef066-d4c9-4695-8120-901c32e2802f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:09.477565 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db06e0049614700a09a866ed046d68c', 'request_id': 'req-b13ef066-d4c9-4695-8120-901c32e2802f'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:09.491843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] The token cache is being invalidate because project 9db06e0049614700a09a866ed046d68c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:40:09.492138 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:09.492827 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:40:09.503000 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1417/2856] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:40:09 2022] DELETE /identity/v3/projects/9db06e0049614700a09a866ed046d68c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:40:19.070959 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:DELETE=9,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:19.487760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:40:57.548135 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:40:57.548655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:40:57.548655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:40:57.567685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] MFA Rules not processed for user `41879b8ab41b49f79ad7f8ae4871f251`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:40:57.568873 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:40:57.623412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1440/2857] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:06.295123 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:06.296753 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:06.353426 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef066-d4c9-4695-8120-901c32e2802f admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:06.356892 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:06.357510 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:06.357786 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:06.358055 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:06.359725 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:06.446670 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:06.448619 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:06.471139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1418/2858] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:41:07.616140 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=25 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:07.641721 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:07.643107 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:07.655542 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4da53d-021b-4250-947b-d04ebd02f347 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:07.659085 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:07.659677 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:07.659955 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:07.660221 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:07.661881 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:07.674841 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:07.676256 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:07.688083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1441/2859] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:16.468298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=63 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:17.682162 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:38.852004 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:38.860766 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2c983-f4d9-41b8-8295-976d0b23a2d0 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:38.864477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:38.865070 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:38.865378 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:38.865648 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:38.867437 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:38.878081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:38.879549 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:38.887824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1419/2860] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:38.964789 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:38.975111 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf4dde-0a67-4053-acf9-1ea653f3045d service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:38.978867 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b94d8-501c-43e4-b76d-c1c6152dcf54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:38.979470 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b94d8-501c-43e4-b76d-c1c6152dcf54 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:38.979743 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b94d8-501c-43e4-b76d-c1c6152dcf54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:38.980007 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b94d8-501c-43e4-b76d-c1c6152dcf54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:38.981833 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-ba2b94d8-501c-43e4-b76d-c1c6152dcf54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:38.995670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b94d8-501c-43e4-b76d-c1c6152dcf54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:38.997105 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b94d8-501c-43e4-b76d-c1c6152dcf54 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:39.008048 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1442/2861] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:48.884169 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=6 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:49.003233 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=6 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:49.060159 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:49.061584 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:49.117290 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9286afd-8e6b-40b1-871a-1607b017d623 service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:49.120796 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:49.121429 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:49.121704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:49.121973 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:49.123649 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:49.184376 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:49.185848 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:49.195247 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1420/2862] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:56.033553 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:56.034077 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:56.034188 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:56.052835 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] MFA Rules not processed for user `770127af5f494b7e9198438441868fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:56.054127 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:56.060866 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1443/2863] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:41:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:59.185435 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.187322 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:59.191552 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=48 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:41:59.264532 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.265519 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b71c2c-ba4a-4149-a911-9a52d655ed39 service cinder] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.265934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:59.272332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.273295 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.273791 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.274294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] PATH_INFO: `/v3/users/41879b8ab41b49f79ad7f8ae4871f251` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.276471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41879b8ab41b49f79ad7f8ae4871f251)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.279098 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.289685 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fed87-c46e-4ca6-9f0e-b19302a4e143 None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.296137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.296755 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.297039 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.297399 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.299447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:59.333020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.334675 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.341193 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41879b8ab41b49f79ad7f8ae4871f251'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.353468 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1444/2864] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:41:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:41:59.353841 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41879b8ab41b49f79ad7f8ae4871f251'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.364324 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41879b8ab41b49f79ad7f8ae4871f251'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.365457 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41879b8ab41b49f79ad7f8ae4871f251'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.365846 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-afaf213f-d9ef-4103-94c7-1af859f5b55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41879b8ab41b49f79ad7f8ae4871f251'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.390890 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1421/2865] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:41:59 2022] DELETE /identity/v3/users/41879b8ab41b49f79ad7f8ae4871f251 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:59.624938 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.637729 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db59bf-fb8a-4042-95cd-4cc1e0642cf1 service nova] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.699358 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.699985 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.700287 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.700524 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] PATH_INFO: `/v3/projects/f797ec18d0a847748116b1dd18bc8cf1` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.701847 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f797ec18d0a847748116b1dd18bc8cf1)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.703686 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.753498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f797ec18d0a847748116b1dd18bc8cf1', 'request_id': 'req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.758828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f797ec18d0a847748116b1dd18bc8cf1', 'request_id': 'req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.768073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f797ec18d0a847748116b1dd18bc8cf1', 'request_id': 'req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.780642 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] The token cache is being invalidate because project f797ec18d0a847748116b1dd18bc8cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:41:59.780926 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.781596 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-2ad3ccdd-30f7-4eef-afb7-703be0dff7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:41:59.783086 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1445/2866] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:41:59 2022] DELETE /identity/v3/projects/f797ec18d0a847748116b1dd18bc8cf1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:41:59.788590 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8db3b-4982-4171-ace1-48a585a31488 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.788874 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8db3b-4982-4171-ace1-48a585a31488 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.789174 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8db3b-4982-4171-ace1-48a585a31488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.809045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-bda8db3b-4982-4171-ace1-48a585a31488 None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:59.810281 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-bda8db3b-4982-4171-ace1-48a585a31488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:59.869946 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1422/2867] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:41:59.878137 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242c5d-613c-4505-8ccc-3a22f0e527ef None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.878426 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242c5d-613c-4505-8ccc-3a22f0e527ef None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.878727 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242c5d-613c-4505-8ccc-3a22f0e527ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.898341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-38242c5d-613c-4505-8ccc-3a22f0e527ef None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:41:59.899545 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-38242c5d-613c-4505-8ccc-3a22f0e527ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:41:59.907252 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1446/2868] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 12:41:59.911908 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8db3b-4982-4171-ace1-48a585a31488 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.921471 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8db3b-4982-4171-ace1-48a585a31488 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.925950 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.926536 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.926812 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.927081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] PATH_INFO: `/v3/domains` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.927722 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 12:41:59.928165 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.929992 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.941207 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1423/2869] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 10 12:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 12:41:59.945447 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242c5d-613c-4505-8ccc-3a22f0e527ef None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:41:59.954756 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242c5d-613c-4505-8ccc-3a22f0e527ef None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:41:59.959987 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:41:59.960590 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:41:59.960869 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:41:59.961169 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:41:59.961840 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:41:59.963740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:41:59.999655 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1447/2870] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 10 12:41:59 2022] POST /identity/v3/projects => generated 397 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:00.004229 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:00.013886 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735980d7-7296-4580-bf6a-289f93b50915 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.017347 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:00.076682 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.077422 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.077707 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.077988 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] PATH_INFO: `/v3/users` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.078629 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.080569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.103837 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1424/2871] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 10 12:42:00 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 12:42:00.108944 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:00.119345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05fa49-eb7f-4aac-baf0-cc6f6127fc97 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.124064 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.124683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.124958 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.125229 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.125920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.127504 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.137877 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1448/2872] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:00.142116 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:00.152765 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec723cb-7b1e-4854-a2d0-29c84dfdc8ca admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.157424 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26ab5f-09e9-4739-b0cf-8b39f65847af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.158017 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26ab5f-09e9-4739-b0cf-8b39f65847af admin admin] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.158294 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26ab5f-09e9-4739-b0cf-8b39f65847af admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.158567 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26ab5f-09e9-4739-b0cf-8b39f65847af admin admin] PATH_INFO: `/v3/roles` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.159205 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26ab5f-09e9-4739-b0cf-8b39f65847af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.160921 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26ab5f-09e9-4739-b0cf-8b39f65847af admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.173298 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1425/2873] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 10 12:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 12:42:00.175498 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:00.185411 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3ea5-c15a-45ae-a37c-862153efc135 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.189984 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.191117 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.191396 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.191662 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] PATH_INFO: `/v3/projects/8b0a879c2c924cb68dd0b20421fb3cf2/users/d8573714341f4ee384f29eaff321b192/roles/ae3c9f4504f84e25bc89e8c30397aedc` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.207526 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0a879c2c924cb68dd0b20421fb3cf2, user_id=d8573714341f4ee384f29eaff321b192, role_id=ae3c9f4504f84e25bc89e8c30397aedc)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.210258 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.219154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:00.227977 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1449/2874] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 10 12:42:00 2022] PUT /identity/v3/projects/8b0a879c2c924cb68dd0b20421fb3cf2/users/d8573714341f4ee384f29eaff321b192/roles/ae3c9f4504f84e25bc89e8c30397aedc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:42:00.236477 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c83531-864c-44f6-9837-639f60a6e3a5 None None] REQUEST_METHOD: `POST` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.236994 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c83531-864c-44f6-9837-639f60a6e3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.237179 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c83531-864c-44f6-9837-639f60a6e3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.267022 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.auth.core [None req-37c83531-864c-44f6-9837-639f60a6e3a5 None None] MFA Rules not processed for user `d8573714341f4ee384f29eaff321b192`. Rule list: `[]` (Enabled: `True`). {{(pid=80064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:00.268248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-37c83531-864c-44f6-9837-639f60a6e3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:00.343074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1426/2875] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:00.354893 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:00.356669 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:00.414029 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339b2b8-12a0-4fc9-9064-08ce22264f57 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:00.417744 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efd62e-0d69-4416-b234-a9195859ea1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d07f733a6c44858ba396c933e370e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:00.418355 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd62e-0d69-4416-b234-a9195859ea1a service cinder] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:00.418635 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd62e-0d69-4416-b234-a9195859ea1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:00.418910 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd62e-0d69-4416-b234-a9195859ea1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:00.428414 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd62e-0d69-4416-b234-a9195859ea1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:00.430397 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd62e-0d69-4416-b234-a9195859ea1a service cinder] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:00.440228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1450/2876] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:00 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:42:08.502214 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c83531-864c-44f6-9837-639f60a6e3a5 None None] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:08.564106 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c83531-864c-44f6-9837-639f60a6e3a5 None None] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:08.569289 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:08.569872 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.570153 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.570483 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] PATH_INFO: `/v3/users/d8573714341f4ee384f29eaff321b192` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.571612 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8573714341f4ee384f29eaff321b192)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:08.573652 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:08.598260 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:08.630658 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8573714341f4ee384f29eaff321b192'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.637625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8573714341f4ee384f29eaff321b192'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.646002 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8573714341f4ee384f29eaff321b192'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.646607 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8573714341f4ee384f29eaff321b192'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.647212 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8573714341f4ee384f29eaff321b192'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:08.665945 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1427/2877] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:42:08 2022] DELETE /identity/v3/users/d8573714341f4ee384f29eaff321b192 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:08.670460 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] REQUEST_METHOD: `POST` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.670948 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.671081 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.690225 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.auth.core [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] MFA Rules not processed for user `474f14ac319744fba24161f36fcf9aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80063) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 12:42:08.691428 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:08.755331 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1451/2878] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 10 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 12:42:08.765449 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:08.766870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:08.823978 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d95f31-1c75-4959-82d1-00f5a87944b4 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:08.827620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:08.828216 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:08.828492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:08.828760 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:08.837625 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:08.839045 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:08.848456 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1428/2879] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:09.108879 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:09.110340 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:09.124093 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd82fa-4e1b-419a-b4e9-d48817a9708f None None] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:09.129073 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:09.129772 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:09.130053 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:09.130324 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] PATH_INFO: `/v3/projects/8b0a879c2c924cb68dd0b20421fb3cf2` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:09.131481 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0a879c2c924cb68dd0b20421fb3cf2)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:09.133620 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:09.180405 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0a879c2c924cb68dd0b20421fb3cf2', 'request_id': 'req-1d595542-08c5-4526-83c2-8a27156934ac'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:09.185870 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0a879c2c924cb68dd0b20421fb3cf2', 'request_id': 'req-1d595542-08c5-4526-83c2-8a27156934ac'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:09.193144 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0a879c2c924cb68dd0b20421fb3cf2', 'request_id': 'req-1d595542-08c5-4526-83c2-8a27156934ac'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:09.204243 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] The token cache is being invalidate because project 8b0a879c2c924cb68dd0b20421fb3cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:42:09.204528 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:09.205151 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:42:09.206631 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1452/2880] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:42:09 2022] DELETE /identity/v3/projects/8b0a879c2c924cb68dd0b20421fb3cf2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:42:11.401003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.402654 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:11.481317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cca0f-45c3-44a9-a855-33a2a45a630d service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.512999 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:11.545469 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.546074 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.546359 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.546641 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.548356 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:11.560032 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.561659 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.572528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1429/2881] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:11.833001 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:11.834362 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:11.862083 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:11.893710 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d595542-08c5-4526-83c2-8a27156934ac admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:11.897341 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:11.897947 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:11.898223 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:11.898495 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:11.900207 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:11.909678 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:11.911157 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:11.919031 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1453/2882] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:15.329040 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:42:15.330709 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:15.344233 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa759bd-3da5-458c-a8fa-bf9770465ece service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:42:15.350464 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '536ca83c26354a77b1c5d263c57f4f7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:42:15.351062 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:42:15.351343 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:42:15.351616 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:42:15.353338 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:42:15.366947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:42:15.368036 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:42:15.378870 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1430/2883] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:42:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:42:21.915465 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=15 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:42:25.374185 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=24 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:16.712781 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:16.714202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:16.729016 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac75f7-f92b-4e8e-9e6a-3d0244c4e5e8 service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:16.733767 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:16.734726 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:16.735202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:16.735568 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:16.738332 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:16.801943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:16.803371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:16.813381 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1454/2884] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:43:19.809492 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:19.811291 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:19.903436 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf119ef-5953-41be-8c1a-1ca43261aee6 service nova] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:19.910317 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:19.910947 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:19.911226 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:19.911528 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:19.913332 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:19.929997 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:19.932445 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:19.946555 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1431/2885] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:43:22.960002 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:22.973676 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81691e1-40d3-4274-b698-29255a9445bf service neutron] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:22.980328 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:22.981020 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:22.981398 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:22.981738 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:22.983626 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:23.013699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:23.013699 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:23.018303 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:23.019839 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:23.026228 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1455/2886] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:43:23.047003 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd996-f0bf-4b6b-a612-ba234428ec86 service placement] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:23.053391 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:23.054248 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:23.054525 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:23.054797 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] PATH_INFO: `/v3/users/94df22112d134130a678f91cae49adef` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:23.056072 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94df22112d134130a678f91cae49adef)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:23.058018 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:23.107490 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94df22112d134130a678f91cae49adef'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.114798 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94df22112d134130a678f91cae49adef'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.121501 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94df22112d134130a678f91cae49adef'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.122117 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94df22112d134130a678f91cae49adef'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.122739 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94df22112d134130a678f91cae49adef'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.140183 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1432/2887] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:23 2022] DELETE /identity/v3/users/94df22112d134130a678f91cae49adef => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 12:43:23.306749 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:23.318920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54beec5c-05d8-4971-97fb-b5604e963cce service placement] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:23.375193 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:23.375817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:23.376091 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:23.376360 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] PATH_INFO: `/v3/projects/65ece09339af406b98e493fa33f2b9c0` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:23.377486 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ece09339af406b98e493fa33f2b9c0)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:23.379378 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:23.424280 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ece09339af406b98e493fa33f2b9c0', 'request_id': 'req-c489051a-e440-4235-a5de-1b916ede1af5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.429828 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ece09339af406b98e493fa33f2b9c0', 'request_id': 'req-c489051a-e440-4235-a5de-1b916ede1af5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.438120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ece09339af406b98e493fa33f2b9c0', 'request_id': 'req-c489051a-e440-4235-a5de-1b916ede1af5'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.449607 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] The token cache is being invalidate because project 65ece09339af406b98e493fa33f2b9c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:23.449888 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.450496 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.451934 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1456/2888] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:23 2022] DELETE /identity/v3/projects/65ece09339af406b98e493fa33f2b9c0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:23.456029 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:23.457448 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:23.470742 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46553c0c-c89d-4c9c-839e-03a8e70718a6 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:23.532314 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:23.532859 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:23.533139 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:23.533450 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] PATH_INFO: `/v3/users/85190e1676ed4086b802d78875c8f7cf` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:23.534620 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=85190e1676ed4086b802d78875c8f7cf)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:23.536543 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:23.538821 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:23.583473 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85190e1676ed4086b802d78875c8f7cf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.590372 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85190e1676ed4086b802d78875c8f7cf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.598626 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85190e1676ed4086b802d78875c8f7cf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.598880 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85190e1676ed4086b802d78875c8f7cf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.599506 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85190e1676ed4086b802d78875c8f7cf'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.616910 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1433/2889] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:43:23 2022] DELETE /identity/v3/users/85190e1676ed4086b802d78875c8f7cf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:23.792577 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:23.794062 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:23.807563 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489051a-e440-4235-a5de-1b916ede1af5 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:23.815652 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:23.867130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:23.867751 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:23.868580 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:23.868846 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] PATH_INFO: `/v3/projects/027a9b17f5e844bc976eaf5db7f51e25` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:23.869970 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=027a9b17f5e844bc976eaf5db7f51e25)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:23.871868 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:23.920533 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027a9b17f5e844bc976eaf5db7f51e25', 'request_id': 'req-5f812906-65e1-46ce-bd0c-df1c1bba4aac'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.926373 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027a9b17f5e844bc976eaf5db7f51e25', 'request_id': 'req-5f812906-65e1-46ce-bd0c-df1c1bba4aac'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.933830 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027a9b17f5e844bc976eaf5db7f51e25', 'request_id': 'req-5f812906-65e1-46ce-bd0c-df1c1bba4aac'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.944923 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] The token cache is being invalidate because project 027a9b17f5e844bc976eaf5db7f51e25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80063) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:43:23.945213 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.945865 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:43:23.947372 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1457/2890] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:43:23 2022] DELETE /identity/v3/projects/027a9b17f5e844bc976eaf5db7f51e25 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:43:33.613005 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:33.941757 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:43:57.619518 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:43:57.620990 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:57.677843 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e5894-7a8a-4133-851a-df10697afa3b admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:43:57.726507 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c4588df9de242e8819de0476f75959c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:43:57.727104 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] REQUEST_METHOD: `GET` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:43:57.727381 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:43:57.727668 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:43:57.729375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:43:57.789569 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:43:57.790993 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:43:57.800882 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1434/2891] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 12:44:03.643141 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:03.644943 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:03.659817 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812906-65e1-46ce-bd0c-df1c1bba4aac admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:03.664155 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:03.664780 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:03.665076 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:03.665374 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] PATH_INFO: `/v3/users/770127af5f494b7e9198438441868fa1` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:03.666507 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] RBAC: Authorizing `identity:delete_user(user_id=770127af5f494b7e9198438441868fa1)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:03.668429 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:03.717529 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770127af5f494b7e9198438441868fa1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:03.727120 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770127af5f494b7e9198438441868fa1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:03.727715 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770127af5f494b7e9198438441868fa1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:03.728371 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770127af5f494b7e9198438441868fa1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:03.736202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770127af5f494b7e9198438441868fa1'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:03.754920 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1458/2892] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:44:03 2022] DELETE /identity/v3/users/770127af5f494b7e9198438441868fa1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:03.930914 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:03.940863 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee0252-345c-4795-bd8c-38bcb32dacec service neutron] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:03.997893 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:03.998511 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:03.998813 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:03.999081 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] PATH_INFO: `/v3/projects/f63c30b5d2af4144b096b78c7ab308a7` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:04.000180 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63c30b5d2af4144b096b78c7ab308a7)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:04.002097 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:04.010109 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=100 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:04.047968 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63c30b5d2af4144b096b78c7ab308a7', 'request_id': 'req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.053313 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63c30b5d2af4144b096b78c7ab308a7', 'request_id': 'req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.059614 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63c30b5d2af4144b096b78c7ab308a7', 'request_id': 'req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.070285 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] The token cache is being invalidate because project f63c30b5d2af4144b096b78c7ab308a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:44:04.070571 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.071197 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.072665 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1435/2893] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:44:03 2022] DELETE /identity/v3/projects/f63c30b5d2af4144b096b78c7ab308a7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:04.076981 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:04.078407 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:04.092142 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17488c77-bf14-4ce3-8494-6b5f544f542d admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:04.149983 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:04.150665 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:04.150898 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:04.151196 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] PATH_INFO: `/v3/users/602fc05fa9fe4aefb4afa003c16cfeb2` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:04.152858 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=602fc05fa9fe4aefb4afa003c16cfeb2)` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:04.154811 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:04.199202 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602fc05fa9fe4aefb4afa003c16cfeb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.205939 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602fc05fa9fe4aefb4afa003c16cfeb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.206551 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602fc05fa9fe4aefb4afa003c16cfeb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.207154 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602fc05fa9fe4aefb4afa003c16cfeb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.214043 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.notifications [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602fc05fa9fe4aefb4afa003c16cfeb2'} {{(pid=80063) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.222301 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:04.233740 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1459/2894] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 10 12:44:04 2022] DELETE /identity/v3/users/602fc05fa9fe4aefb4afa003c16cfeb2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:04.398309 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Authenticating user token {{(pid=80064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:04.399824 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.fernet_utils [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:04.415091 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a279-a75b-4c6f-a12a-55a0feaa3026 admin admin] Validating token access rules against request {{(pid=80064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:04.475149 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474f14ac319744fba24161f36fcf9aa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd2efc376f4648d0a2b40e0c73cde3c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:04.475704 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:04.475979 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:04.476250 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] PATH_INFO: `/v3/projects/79e32c4ab8954579906632ff27e2c999` {{(pid=80064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:04.477375 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e32c4ab8954579906632ff27e2c999)` {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:04.479279 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] RBAC: Authorization granted {{(pid=80064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:04.523265 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e32c4ab8954579906632ff27e2c999', 'request_id': 'req-2449b5e2-64a7-4f19-a1eb-036e27edf3db'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.529407 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e32c4ab8954579906632ff27e2c999', 'request_id': 'req-2449b5e2-64a7-4f19-a1eb-036e27edf3db'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.536710 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e32c4ab8954579906632ff27e2c999', 'request_id': 'req-2449b5e2-64a7-4f19-a1eb-036e27edf3db'} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.547689 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] The token cache is being invalidate because project 79e32c4ab8954579906632ff27e2c999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 12:44:04.547986 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.548608 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG keystone.notifications [None req-2449b5e2-64a7-4f19-a1eb-036e27edf3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 12:44:04.550164 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: [pid: 80064|app: 0|req: 1436/2895] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 10 12:44:04 2022] DELETE /identity/v3/projects/79e32c4ab8954579906632ff27e2c999 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 12:44:14.233089 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:14.544696 dv-cinder-852261-3-iscsi devstack@keystone.service[80064]: DEBUG dbcounter [-] [80064] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 12:44:27.337345 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Authenticating user token {{(pid=80063) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 12:44:27.339365 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:27.406765 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeed3d5-afd3-431e-a1d3-3f424b2d08f7 admin admin] Validating token access rules against request {{(pid=80063) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 12:44:27.465520 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60266105-5ef3-41de-8a91-ea85fee117d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18b7024ea0ae4deeba88ebd2f8d1f909', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab58563f96aa4f1f8b29f3a3f19ca6b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80063) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 12:44:27.466130 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60266105-5ef3-41de-8a91-ea85fee117d2 service placement] REQUEST_METHOD: `GET` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 12:44:27.466412 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60266105-5ef3-41de-8a91-ea85fee117d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 12:44:27.466695 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60266105-5ef3-41de-8a91-ea85fee117d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80063) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 12:44:27.468670 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.fernet_utils [None req-60266105-5ef3-41de-8a91-ea85fee117d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80063) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 12:44:27.480347 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60266105-5ef3-41de-8a91-ea85fee117d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 12:44:27.481973 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60266105-5ef3-41de-8a91-ea85fee117d2 service placement] RBAC: Authorization granted {{(pid=80063) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 12:44:27.491683 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: [pid: 80063|app: 0|req: 1460/2896] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 10 12:44:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 12:44:37.487483 dv-cinder-852261-3-iscsi devstack@keystone.service[80063]: DEBUG dbcounter [-] [80063] Writing DB stats keystone:SELECT=58 {{(pid=80063) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}